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CALL  FOR  PAPERS 

SPECIAL  ISSUE  ON 

MILITARY  OPERATIONS  RESEARCH  METHODS 

FOR 

FUTURE  R&D  CONCEPT  EVALUATION 

Limited  research  and  development  (R&D)  budgets  make  it  imperative  that  the 
United  States  analyze  the  potential  operational  benefits  of  future  system  concepts  and 
select  the  most  promising  concepts  for  further  R&D  spending.  Military  operations 
research  offers  several  techniques  to  help  senior  DoD  decision-makers  prioritize  future 
R&D  concepts.  The  purpose  of  this  special  issue  is  to  describe  the  most  effective 
techniques  in  use  and  to  propose  improvements  to  military  R&D  concept  evaluation 
techniques. 

Since  many  of  the  DoD  R&D  concept  evaluation  studies  are  classified,  we  are 
willing  to  publish  the  unclassified  techniques  or  the  techniques  with  notional  data. 
However,  in  accordance  with  our  editorial  policy,  we  require  certification  from  a  senior 
decision-maker  that  the  military  R&D  concept  evaluation  techniques  were  used. 

Interested  authors  should  submit  abstracts  by  November  30, 1996  and  papers 
by  January  31, 1997.  The  papers  should  be  submitted  in  accordance  with  our  current 
editorial  policy.  All  papers  will  be  refereed. 

We  are  also  seeking  volunteers  to  serve  as  guest  editors,  associate  editors,  and 
referees  for  this  special  issue. 

Please  contact  me  if  you  are  interested  in  authoring  a  paper  or  serving  as  an 
editor/referee  for  this  special  issue. 


DR.  GREGORY  S.  PARNELL 

Editor,  Military  Operations  Research 

Department  of  Mathematical  Sciences 

Virginia  Commonwealth  University 

Oliver  Hall,  1015  W.  Main  Street 

P.O.  Box  842014 

Richmond,  VA  23284-2014 

Phone:  804-828-1301,  ext.  133/Fax:  804-828-8785 

Email:  gpamell@vcu.edu  and  gspamell@aol.com 


Military  Operations  Research:  What's  Changed  and  What  Hasn't? 

Gregory  S.  Parnell,  Editor,  Military  Operations  Research 

What  hasn't  changed? 

We  want  to  maintain  and  improve  upon  the  high  quality  of  articles  published  in  Military 
Operations  Research.  The  first  Editor,  Dr.  Peter  Purdue,  and  his  Associate  Editors  have  done  a 
great  job  of  carefully  reviewing  and  selecting  outstanding  articles.  The  new  editorial  board 
and  1  will  continue  the  high  standards  they  have  set. 

What  has  changed? 

The  editorial  policy  has  changed.  We  have  developed  procedures  and  instructions  to 
authors  that  will  expedite  the  review  and  publication  process. 

Our  new  editorial  policy  (see  below)  requests  that  authors  identify  the  value  of  their 
analysis  or  research  effort  described  in  their  paper.  Authors  must  submit  a  statement  of  con¬ 
tribution  and,  for  application  articles,  a  letter  from  a  decision-maker  stating  the  benefits  of 
the  analysis  or  research. 

The  articles  submitted  to  the  journal  cover  many  military  operations  research  problem 
domains  and  methodologies.  In  order  to  assign  the  most  appropriate  reviewer,  we  have 
identified  application  areas  and  methodologies.  We  have  also  expanded  the  number  of 
Associate  Editors  to  insure  we  have  expertise  in  all  of  these  areas.  In  addition,  we  have 
developed  procedures  to  insure  timely  review  of  submitted  papers.  To  help  expedite  the 
publication  process,  we  have  developed  instructions  for  Military  Operations  Research  authors 
(see  below). 


Editorial 

Policy 

and 

Submission 
of  Papers 


EDITORIAL  POLICY 

The  title  of  our  journal  is  Military  Operations  Research.  We  are  interested  in  publishing 
articles  that  describe  operations  research  (OR)  methodologies  used  in  important  military  appli¬ 
cations.  We  specifically  invite  papers  that  are  significant  military  applications  of  OR  method¬ 
ologies.  Of  particular  interest  are  papers  that  present  case  studies  showing  innovative  OR 
applications,  apply  OR  to  major  policy  issues,  introduce  interesting  new  problem  areas, 
highlight  educational  issues,  and  document  the  history  of  military  OR.  Papers  should  be 
readable  with  a  level  of  mathematics  appropriate  for  a  master's  program  in  OR. 

All  submissions  must  include  a  statement  of  the  major  contribution.  For  applications 
articles,  authors  are  requested  to  submit  a  letter  to  the  Editor — exerpts  to  be  published  with 
the  paper — ^from  a  senior  decision-maker  (government  or  industry)  stating  the  benefits 
received  from  the  analysis  described  in  the  paper. 

To  facilitate  the  review  process,  authors  are  requested  to  categorize  their  articles  by 
application  area  and  OR  methodology,  as  described  by  the  following  lists.  Additional  cate¬ 
gories  may  be  added.  (We  use  the  MORS  working  groups  as  our  applications  areas  and  our 
list  of  methodologies  are  those  typically  taught  in  most  graduate  programs.) 


INSTRUCTIONS  TO  MILITARY  OPERATIONS  RESEARCH  AUTHORS 

The  purpose  of  the  "instructions  to  Military  Operations  Research  authors"  is  to  expedite 
the  review  and  publication  process.  If  you  have  any  questions,  please  contact  Mr.  Michael 
Cronin,  MORS  Editorial  Assistant  (email:  morsoffice@aol.com). 
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Authors  should  submit  their  manuscripts  (3  copies)  to: 

Dr.  Gregory  S.  Parnell,  Editor,  Military  Operations  Research 
Military  Operations  Research  Society 
101  South  Whiting  Street,  Suite  202 
Alexandria,  VA  22304 

The  manuscript  should  have  camera  ready  illustrations  and  an  electronic  version  of  the 
manuscript  prepared  in  WordPerfect  or  Microsoft  Word.  Per  the  editorial  policy,  please 
provide: 

•  authors  statement  of  contribution  (briefly  describe  the  major  contribution  of  the  article) 

•  letter  from  senior  decision-maker  (application  articles  only) 

•  military  OR  application  area(s) 

•  OR  methodology  (ies) 
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EDITORIAL  POLICY  AND  SUBMISSION  OF  PAPERS 


Length  of  Papers 

Submissions  will  normally  range  from  5-25  pages  (double  spaced,  12  pitch,  including 
illustrations).  Exceptions  will  be  made  for  applications  articles  submitted  with  a  senior 
decision-maker  letter  signed  by  the  Secretary  of  Defense. 

Figures,  Graphs  and  Charts 

Please  include  camera-ready  copies  of  all  figures,  graphs  and  charts.  The  figure  should 
be  of  sufficient  size  for  the  reproduced  letters  and  numbers  to  be  legible.  Each  illustration 
must  have  a  caption  and  a  number  which  orders  the  placement  of  the  illustration. 

Mathematical  and  Symbolic  Expressions 

Authors  should  put  mathematical  and  symbolic  expressions  in  WordPerfect  or 
Microsoft  Word  equations.  Lengthy  expressions  should  be  avoided. 

Approval  of  Release 

All  submissions  must  be  unclassified  and  be  accompanied  by  release  statements  where 
appropriate.  By  submitting  a  paper  for  review,  an  author  certifies  that  the  manuscript  has 
been  cleared  for  publication,  is  not  copyrighted,  has  not  been  accepted  for  publication  in  any 
other  publication,  and  is  not  under  review  elsewhere.  All  authors  will  be  required  to  sign  a 
copyright  agreement  with  MORS. 

Abbreviations  and  Acronyms 

Abbreviations  and  acronyms  (A&A)  must  be  identified  at  their  first  appearance  in  the 
text.  The  abbreviation  or  acronym  should  follow  in  parentheses  the  first  appearai\ce  of  the 
full  name.  To  help  the  general  reader,  authors  should  minimize  their  use  of  acronyms.  A  list 
of  acronyms  should  be  provided  with  the  manuscript. 

Footnotes 

We  do  not  use  footnotes.  Parenthetical  material  may  be  incorporated  into  a  notes  section 
at  the  end  of  the  text,  before  the  acknowledgment  and  references  sections.  Notes  are 
designated  by  a  superscript  letter  at  the  end  of  the  sentence. 

References 

References  should  appear  at  the  end  of  the  paper  and  be  unnumbered  and  listed  in 
alphabetical  order  by  the  name  of  the  first  author. 


POTENTIAL  PAPERS  OR  SUGGESTIONS  FOR  THE  JOURNAL 

Military  Operations  Research  is  your  journal.  I  need  your  help  to  identify  the  best  articles 
for  submission  to  the  journal!  If  you  have  questions  about  a  potential  paper  or  suggestions 
for  articles,  please  send  me  e-mail  at  gsparnell@aol.com. 

Tm  looting  forward  to  seeing  your  article  in  Military  Operations  Researchl 
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The  book  is  comprised  of  seven  sections: 

(1)  Theater-Level  Modeling 

(2)  Mathematical  Models  of  Combat 

(3)  Historical  Analysis 

(4)  Weapon  System  Analysis 

(5)  Command,  Control,  Communications  and 
Intelligence 

(6)  Cost  Effectiveness  Analysis 

(7)  Modeling  in  Support  of  Operations  Desert 
Shield  and  Desert  Storm 


ORDER  FORM 

Warfare  Modeling 

Published  by  the 

MILITARY  OPERATIONS  RESEARCH  SOCIETY 
in  cooperation  with  John  Wiley  &  Sons 
600  pages,  $35. 

The  Military  Operations  Research  Society  (MORS) 
serves  the  students,  developers,  practitioners  and 
clients  of  military  operations  research  through  the 
ongoing  production  of  high-quality,  low-cost  meetings, 
books  and  periodicals. 

Warfare  Modeling  was  selected  for  publication  by 
MORS  because  it  contains  25  state-of-the-art  chapters, 
addressing  a  wide  class  of  operations  research  models 
that  are  of  central  importance  in  military  planning, 
analysis  and  operations.  Research  on  this  subject 
matter  seldom  appears  in  the  open  literature.  Most  of  it 
appears  in  reports  and  documents  of  government 
agencies  and  advisory  corporations.  This  book  is  an 
invaluable  reference  for  military  OR  professionals  as 
well  as  for  a  more  general  audience  of  researchers  and 
practitioners.  The  scope  of  models  included — analytic 
and  simulative,  stochastic  and  deterministic,  simple 
and  complex,  applied  and  theoretical,  domestic  and 
international — is  very  broad. 

Warfare  Modeling  was  edited  by  Jerome  Bracken  of 
Yale  University,  Moshe  Kress  of  Israel’s  Center  for 
Military  Analyses,  and  Richard  E.  Rosenthal  of  the  US 
Naval  Postgraduate  School.  The  Foreword  is  by  Wayne 
P.  Hughes,  Jr.,  a  retired  Navy  captain  and  MORS 
Fellow. 


Send  orders  to:  Military  Operations  Research  Society  (MORS) 
101  S.  Whiting  Street,  Suite  202,  Alexandria,  VA  22304 
703-751-7290;  FAX  703-751-8171;  e-mail  morsoffice@aolcom 


Please  send _ copies  of  Warfare  Modeling  . @$35.00  each 

Freight:  U.S . @  $5-00  each 

Canada  . @  $8.00  each 

Other  International  . @$18.00  each 
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MILITARY  OPERATIONS  RESEARCH:  BOMB  DAMAGE  ASSESSMENT 
RESPONDING  TO  CHANGE  AND  SORTIE  REQUIREMENTS 


hy  LtCol  Paul  F.  Auclair 
Edward  F.  Mykytka 
and  Dr.  Gregory  S.  Parnell 

For  the  61st  Military  Operations 
Research  Symposium,  the  Program  Chair, 
Dan  Barker,  developed  a  survey  to  assess 
how  the  military  operations  research  (MOR) 
community  was  responding  to  change.  The 
survey  was  mailed  in  late  1992  (three  years 
after  the  fall  of  the  Berlin  Wall)  and  over 
1200  of  the  3500  surveys  were  returned. 
After  the  symposium,  Paul  Auclair,  Ed 
Mykytka,  and  Greg  Parnell  analyzed  the 
survey  results  for  the  journal.  The  article 
provides  a  descriptive  summary  of  the 
membership,  assesses  the  need  for  profes¬ 
sional  education,  and  identifies  key  insights 
about  how  the  focus  of  the  community  had 
changed  by  late  1992. 


BENCHMARKING  AND  EFFICIENT 
PROGRAM  DELIVERY  FOR  THE 
DEPARTMENT  OF  DEFENSE’S 
BUSINESS-LIKE  ACTIVITIES 

by  Dr.  William  F.  Bowlin 

In  recent  years,  the  concept  of  total  qual¬ 
ity  management  (TQM)  has  been  infused 
into  many  government  operations.  A  key 
component  of  TQM  is  benchmarking. 
William  F.  (Bud)  Bowlin  illustrates  an  inter¬ 
nal  benchmarking  approach  using  a 
methodology  called  data  envelopment 
analysis  (DEA)  and  Department  of  Defense 
commissary  data.  By  using  DEA,  an  analyst 
is  able  to  identify  superior  performance, 
quantify  performance  gaps,  and  contribute 
to  continuous  improvement. 


by  Dennis  K.  Evans 

Computer  programs  that  simulate  war¬ 
fare  generally  do  not  model  bomb  damage 
assessment  (BDA)  in  a  complex  fashion. 
Most  models  either  assume  perfect  BDA  or 
no  BDA,  or  allow  the  user  to  select  between 
these  two  options.  Assuming  perfect  BDA 
will  allow  a  given  force  to  destroy  a  given 
target  set  much  more  rapidly  than  will  be 
the  case  if  there  is  no  BDA.  It  would  superfi¬ 
cially  appear  that  perfect  BDA  and  no  BDA 
are  opposite  limiting  extremes.  This  is  not 
the  case.  The  opposite  limiting  extremes  are 
actually  perfect  BDA  and  extremely  bad 
BDA.  "No  BDA"  is  an  intermediate  case. 
Hence,  it  is  probably  more  reasonable  to 
assume  no  BDA  than  perfect  BDA  in  doing 
computer  modeling.  It  is  easy  to  simulate 
perfect  BDA  or  nonexistent  BDA,  but  any 
method  of  modeling  imperfect  BDA  will  be 
highly  dependent  on  the  scenario  assumed. 
This  suggests  that  "no  BDA"  may  be  prefer¬ 
able  to  "flawed  BDA"  as  an  assumption  in 
computer  modeling. 
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MODELING  COSTAND 
SCHEDULE  UNCERTAINTIES— 

A  WORK  BREAKDOWN 
STRUCTURE  PERSPECTIVE 

by  Paul  R.  Garvey 

When  cost  and  schedule  uncertainty 
analyses  are  presented  to  decision-makers, 
questions  asked  with  increased  urgency  are: 
What  is  the  likelihood  of  achieving  cost  and 
schedule?  What  is  the  chance  of  exceeding 
the  most  likely  cost  for  a  given  schedule? 
How  are  cost  reserve  recommendations  dri¬ 
ven  by  schedule  uncertainties?  Questions 
such  as  these  are  not  readily  answered  by 
current  analysis  tools  and  techniques.  This 
paper  advances  the  state-of-the-practice  by 
describing  a  family  of,  less  known,  multi¬ 
variate  probability  models  suitable  for 
addressing  these  and  related  questions.  The 
application  of  these  models  is  presented 
from  a  work  breakdown  structure  perspec¬ 
tive,  the  traditional  framework  for  develop¬ 
ing  program  resource  estimates. 
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DYNAMICAL  INSTABILITY  IN  COMBAT 
MODELS:  COMPUTER  ARITHMETIC 
AND  MATHEMATICAL  MODELS  OF 
AHRITION  AND  REINFORCEMENT 

by  Dr.  Julian  1.  Palmore 

Foremost,  this  paper  reports  a  methodology 
that  can  be  used  to  assess  information  gain  and 
the  value  of  information  in  combat  operations. 
Secondly,  the  experiments  discussed  in  this 
report  give  modest  insight  into  underlying  rela¬ 
tionships  between  tactical  intelligence  informa¬ 
tion  and  combat  results.  Finally,  the  paper 
reports  a  unique  application  of  information  theo¬ 
ry  to  measure  intelligence  information  concern¬ 
ing  the  size  and  disposition  of  enemy  forces. 

Combat  models,  nonlinear  deterministic 
models  of  decision  making  processes,  that  deal 
with  attrition  of  opposing  forces,  may  contain 
dynamical  instabilities  and  structural  variance. 
Chaos  in  computation  is  one  cause  of  instabili¬ 
ties  in  computer  simulations  of  combat.  Another 
cause  of  instabilities  in  decision  processes  is 
attributed  to  timing  problems  that  arise  when 
thresholds  are  crossed.  An  example  of  computer 
arithmetic  effects  for  Patriot  missile  software  is 
given  in  which  very  small  timing  errors  accumu¬ 
late  and  cause  gross  errors  in  detection  and  rang¬ 
ing.  Several  simple  mathematical  models  of 
attrition  and  reinforcement  are  given  and  ana¬ 
lyzed  for  chaotic  behavior  and  nonlinear  effects. 
Nonmonotonicity  is  demonstrated  in  the 
response  of  battle  outcomes  to  changes  in 
resources. 


MILITARY  TRAINING  RESOURCE 
SCHEDULING:  SYSTEM  MODEL, 
OPTIMAL  AND  HEURISTIC  DECISION 
PROCESSES 

by  LTC  Mike  McGinnis, 

Professor  Emmanuel  Fernandez-Gaucherand 
and  Dr.  Pitu  B.  Mirchandani 

The  United  States  Army  trains  thousands  of 
new  soldiers  each  year  to  fill  vacancies  in  Army 
organizations.  Initial  entry  training  consists  of 
two  sequential  phases:  Basic  Combat  Training 
followed  by  Advanced  Individual  Training. 
Until  recently,  manual  heuristic  methods  were 
used  to  schedule  hundreds  of  training  compa¬ 
nies  for  initial  entry  training.  We  present  two 
approaches  for  scheduling  training  resources  for 
the  Basic  Combat  Training  phase  of  initial  train¬ 
ing.  One  is  a  dynamic  programming  decision 
model  for  optimally  scheduling  training 
resources.  The  second  is  a  heuristic  procedure 
implemented  in  an  decision  support  system 
(DSS).  Computational  results  are  given  for  both 
approaches.  The  heuristic  provides  "good"  solu¬ 
tions  in  terms  of  three  performance  measures: 
training  quality  as  measured  by  an  instructor-to- 
trainee  ratio,  resource  utilization,  and  training 
costs. 
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ABSTRACT 

his  paper  provides  a  summary  of  some 
top  level  insights  obtained  from  analy¬ 
sis  of  a  survey  of  military  operations 
research  practitioners  conducted  by  the 
Military  Operations  Research  Society  in  late 
1992  through  early  1993.  In  this  paper,  data 
obtained  from  the  survey  is  used  to  provide 
a  descriptive  summary  of  the  military  opera¬ 
tions  research  community  and  how  it  was 
responding  to  changes  in  the  military  and 
political  environments  at  the  time  the  survey 
was  conducted. 

INTRODUCTION 

In  preparation  for  the  61st  Military 
Operations  Research  Society  (MORS) 
Symposium,  military  operations  research 
practitioners  were  surveyed  in  late  1992 
through  early  1993  to  assess  how  the  mili¬ 
tary  operations  research  (MOR)  community 
was  responding  to  change.  Of  approxi¬ 
mately  3,500  surveys  distributed,  1,240  were 
completed  and  returned.  The  responses  con¬ 
tain  a  great  deal  of  information  about  the 
analysts  who  replied,  the  types  of  analyses 
they  performed,  and  the  organizations  they 
worked  for. 

The  survey  data  could  easily  support 
analysis  of  many  detailed  questions  and 
comparisons.  An  initial  summary  was  pre¬ 
sented  at  the  symposium  held  at  the  Air 
Force  Institute  of  Technology  in  June  1993  by 
Mr.  Dan  Barker,  the  symposium  chair.  The 
purpose  of  this  paper  is  to  highlight  some  of 
the  top-level  insights  gleaned  from  analysis 
of  the  survey  data. 

THE  SURVEY 

The  survey  was  written  and  distributed 
by  Barker  in  support  of  the  symposium 
theme,  "Responding  to  Change."  It  contains 
17  questions  concerning  the  demographics 
of  the  respondents,  the  nature  of  their  work, 
and  their  perceptions  of  organizational 
changes  and  world  events. 

(The  survey  is  included  in  the  Appen¬ 
dix.  To  obtain  a  electronic  copy  of  the 
response  data  [which  can  be  read  either  as  a 
dBase  or  Microsoft  Excel  file],  send  your 
request  to  MORS  at  the  address  indicated  on 
the  inside  front  cover  of  this  journal.) 


THE  SURVEY  POPULATION 

The  survey  was  distributed  to  those 
individuals  who  were  members  of  MORS 
during  1992.  One  becomes  a  member  of 
MORS  by  attending  any  one  of  its  scheduled 
meetings.  Attendance  at  a  meeting  assures 
membership  in  the  society  for  three  years. 
Thus,  those  individuals  who  attended  at 
least  one  MORS-sponsored  meeting  since 
1989  were  mailed  a  survey.  Although  the 
respondents  represent  a  sizable  proportion 
(35%)  of  the  MORS  membership,  they  do 
not  necessarily  reflect  the  larger  population 
of  military  operations  research  practitioners 
who,  for  one  reason  or  another,  did  not  par¬ 
ticipate  in  MORS-sponsored  activities  in  the 
preceding  three  years. 

Figure  1  compares  the  proportions  of  the 
MORS  population  affiliated  with  various 
agencies  to  the  proportions  exhibited  by  the 
survey  respondents.  As  seen  in  this  figure, 
the  proportions  are  comparable.  In  fact, 
the  proportions  differ  by  less  than  five 
percentage  points  for  any  affiliation  category. 


Figure  1:  Survey  Respondents  vs. 
MORS  Membership 


Figure  2  illustrates  that  the  survey 
respondents  were  a  mix  of  private  sector, 
civil  service,  and  military  analysts.  Private 
sector  civilians  accounted  for  38  percent  of 
the  responses,  civil  service  employees  made 
up  39  percent,  and  military  analysts  round¬ 
ed  out  the  final  23  percent.  Unfortunately, 
we  were  unable  to  compare  the  proportions 
presented  in  Figure  2  with  those  of  the 
MORS  membership  as  a  whole  since  the 
latter  were  not  available  from  the  society. 
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Figure  2:  Affiliations  of  Respondents 


Together,  Figures  1  and  2  suggest  that  the 
survey  population  fairly  represented  the  various 
agencies  that  comprised  the  MORS  membership 
in  early  1993  and  that  no  single  constituency 
dominated  the  responses.  Therefore,  we  would 
expect  the  survey  responses  to  be  representative 
of  the  opinions,  attitudes,  and  characteristics  of 
the  MO^  population. 

THE  ANALYSTS 
Experience 


<5  6-10  11-15  16-20  21-25  >25 


Years 


Table  1  and  Figure  3  depict  the  distribution 
of  analytical  experience  for  the  respondents  in 
five-year  increments. 


Table  1:  Years  of  Experience  in  Analysis 


Years  of 
Experience 

Number  of  Respondents 

Military 

Civil 

Service 

Private 

Total 

>25 

0 

93 

94 

187 

21-25 

5 

85 

87 

177 

16-20 

19 

80 

74 

173 

11-15 

44 

67 

103 

214 

6-10 

67 

91 

71 

229 

<5 

110 

47 

41 

198 

Overall 

245 

463 

470 

1178 

Figure  3:  Analysis  Experience 

Although  experience  levels  are  distributed 
fairly  evenly  in  the  aggregate  (with  roughly  the 
same  number  in  each  five-year  group),  there  are 
substantial  differences  by  affiliation.  Likely 
because  military  careers  are  relatively  short, 
nearly  half  of  the  military  analysts  had  less  than 
5  years  of  analytical  experience,  90  percent  had 
less  than  15  years  experience,  while  those  with 
over  20  years  of  analytical  experience  were  rare. 
In  contrast,  a  relatively  small  proportion  of  civil¬ 
ian  analysts  had  less  than  5  years  of  experience, 
and  a  relatively  large  number  had  well  over  20 
years  of  such  experience. 

Regardless  of  affiliation,  however,  the  vast 
majority  of  respondents  had  spent  most  of  their 
analytical  careers  focused  on  military  operations 
research,  as  shown  in  Table  2. 


Page  10 


Military  Operations  Research,  Spring  1996 


RESPONDING  TO  CHANGE 


Table  2:  Military  vs.  Analytical  Experience 

Years  of  Years  of  Analytical  Experience 

Military - 

OR  Exp.  1-5  6-10  11-15  16-20  21-25  >25 

>25  10^ 

21-25  96  28 

16-20  109  34  28 

11-15  130  36  26  18 

6-10  190  55  21  18  11 

1-5  208  47  30  14  8  3 

OveraU  |  208  237  215  180  182  190 

Figure  4;  Education  Levels 

Education 

Self-Classification 

The  respondents  reported  a  high  level  of 

education,  with  over  80  percent  claiming  at  least  Figure  5  depicts  how  the  survey  respon- 

a  master's  degree.  Figure  4  depicts  the  propor-  dents  indicated  that  they  would  describe  their 

tions  of  population  subgroups  with  various  jobs  to  those  unfamiliar  with  analysis.  It  is  inter¬ 
levels  of  education.  Note  the  contrast  in  the  dis-  esting  to  note  the  diversity  of  alternatives  used 

tributions  of  degrees  between  the  military  and  by  respondents  to  describe  their  discipline, 

civilian  groups.  The  military  group  has  a  slightly  Although  the  category  "Systems  Analyst"  scored 

higher  proportion  of  advanced  degrees,  but  the  the  highest,  it  did  not  constitute  a  majority  of  the 

proportion  of  civilians  with  doctoral  degrees  is  responses.  In  fact,  the  second  largest  response 

more  than  double  that  of  the  military.  We  suspect  was  "Other,"  perhaps  suggesting  that  other  cate- 

that  the  value  that  the  military  places  on  gradu-  gories,  such  as  "Cost  Analyst"  or  "Economist," 

ate  education  tends  to  explain  the  higher  propor-  should  have  been  included, 

tion  of  military  members  with  master's  degrees  The  results  imply  that  military  OR  commu- 

while  the  shorter  careers  and  applied  focus  of  nity  has  retained  its  original  interdisciplinary 
military  analysts  might  account  for  the  smaller  nature-to  the  point  that  it  is  not  dominated  by 
proportion  of  military  members  with  doctorates.  any  particular  academic  or  professional  affilia¬ 
tion.  They  could  also  imply  that,  being  rather 


Figure  5:  Alternate  Job  Descriptions 
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complex,  interdisciplinary,  and  not  popularly 
understood,  military  operations  research  is  diffi¬ 
cult  to  describe  tersely  to  those  not  familiar  with 
the  discipline. 

Training  Needs  &  Activities 

Despite  the  very  high  educational  level 
observed,  about  half  of  the  respondents  felt  they 
needed  further  education  or  training.  The  per¬ 
ceived  need  for  additional  training  in  analytical 
and  computer  techniques  appeared  to  be  a  func¬ 
tion  of  organizational  affiliation.  Figure  6  dis¬ 
plays  the  proportion  of  respondents  that 
indicated  a  need  for  such  training  within  each 
organizational  affiliation,  ordered  by  the  overall 
proportion  expressing  a  need  for  further  train¬ 
ing.  Overall,  about  58  percent  of  the  military 
analysts  reported  a  perceived  need  for  additional 
training  in  analytical  and  computer  techniques; 
only  about  38  percent  of  non-government 
civilians  perceived  need  for  such  training. 
Government  civilians  fell  between  the  two 
groups. 

As  seen  in  Figure  7,  educational  level  related 
to  the  perceived  need  for  additional  training  in 
analytical  techniques.  Not  surprisingly,  the  high¬ 
er  the  degree  level,  the  lower  the  perceived  need 
for  further  education  or  training  in  analytical 
techniques.  On  the  other  hand,  educational  level 
had  less  impact  on  the  perceived  need  for  addi¬ 
tional  training  in  computer  techniques.  As  seen 


Bochelors  Ma$t©rs  Ooctorat© 

Education  Levet 


Figure  7:  Perceived  Training  Needs  by 
Education  Level 

in  Figure  7,  the  proportion  of  individuals 
expressing  a  need  for  additional  training  in  com¬ 
puter  techniques  was  approximately  the  same 
for  those  with  either  bachelors  or  masters 
degrees. 

Of  those  who  felt  they  needed  additional 
training,  a  sizable  proportion  (60%)  was  actively 
engaged  in  self  study.  Nearly  a  fifth  (18%)  of  the 
respondents  were  currently  enrolled  in  formal 
university  courses.  Overall,  it  seems  that  the 
respondents  took  education  seriously,  placed  a 
high  value  on  it,  and  expended  the  resources 
necessary  to  obtain  it.  This  would  seem  to  sug¬ 
gest  that  the  MOR  community  was,  on  an  indi¬ 
vidual  level  at  least,  attempting  to  respond  to 
technical  changes  in  the  field. 


Figure  6:  Perceived  Training  Needs  by  Affiliation 


Page  12 


Military  Operations  Research,  Spring  1996 


RESPONDING  TO  CHANGE 


ANALYSIS 

Functional  Area  Emphasis 

To  provide  a  broad  perspective  on  where  the 
MOR  community  perceived  that  it  was,  and 
expected  it  would  be,  investing  much  of  its  ana¬ 
lytical  effort.  Figure  8  depicts  the  areas  of  prima¬ 
ry  analytical  emphasis  of  the  respondents 
(without  regard  to  organizational  affiliation) 
during  the  periods  1982-87,  1988-92,  and 
projected  for  1993-97, 

The  analysis  of  system  or  individual  perfor¬ 
mance  tends  to  dominate  all  other  areas.  Note, 
however,  that  campaign  analyses  are  broken  into 
conventional  and  strategic  components  and, 
taken  together,  represent  a  healthy  interest  in 
force  effectiveness  studies. 

Of  additional  interest  are  those  categories 
that  are  not  primary  areas  of  emphasis.  For  all 
periods  considered.  Operations  &  Maintenance 
(O&M),  readiness,  and  sustainability  have  not 
been  the  primary  area  of  emphasis  of  many  ana¬ 
lysts.  We  can  infer  that  those  areas  have  clearly 
not  constituted  the  most  important  issues 
addressed  by  the  MOR  community.  We  should 
be  careful  not  to  conclude  that  those  areas  have 
not  received  sufficient  attention.  The  survey  does 
not  address  the  appropriateness  of  the  level  of 
effort  invested  in  each  area;  it  only  identifies  the 
proportion  of  analysts  who  consider  particular 
areas  as  dominant  in  their  own  work.  Never¬ 
theless,  those  less  emphasized  areas  might  have 


a  great  deal  to  do  with  our  warfighting  capa¬ 
bility  and  interoperability.  The  level  of  effort 
afforded  them  might  warrant  reconsideration. 

Theater  Emphasis 

While  the  primary  areas  of  emphasis  indi¬ 
cated  a  greater  focus  on  campaign  and  force 
structure  cost  analysis,  there  was  less  effort 
devoted  to  specific  theaters  of  operation.  Figure 
9  depicts  the  primary  theaters  of  analysis  among 
the  respondents  during  the  periods  1982-87, 
1988-92,  and  projected  for  1993-97. 

This  figure  makes  it  very  clear  that,  regard¬ 
less  of  time  period  considered,  most  analysts  do 
not  focus  on  a  primary  theater.  For  analysts 
working  primarily  on  a  particular  theater,  there 
has  been  a  noticeable  decrease  in  those  focusing 
on  the  Central  European  theater.  Not  surprising¬ 
ly,  there  has  been  much  more  work  in  the 
Southwest  Asia  theater  in  recent  years.  On  the 
other  hand,  despite  events  in  Somalia  and 
Eastern  Europe,  those  theaters  had  attracted  rela¬ 
tively  little  attention  at  the  time  the  survey  was 
administered. 

Potential  Near-Term  Threats 

Figure  10  displays  the  survey  respondents' 
perceptions  of  the  severity  of  various  threats  to 
national  security  for  the  five  year  period  begin¬ 
ning  in  1993.  The  threats  were  classified  as 
"extremely  serious"  or  as  "one  of  the  two  most 
serious." 


Figure  8:  Primary  Areas  of  Analysis 
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Figure  9:  Primary  Theater  of  Analysis 


The  most  serious  perceived  threat  to  US 
national  security  was  third  world  use  of  nuclear 
weapons,  followed  by  our  own  federal  budget 
deficit.  It  is  interesting  to  observe  the  mix  of 
purely  military  and  social  threats  to  national 
security.  If  MOR  analysis  were  to  respond  to  all 
perceived  threats  to  national  security,  members 
of  the  MOR  community  could  very  likely  be 
expected  to  study  and  analyze  issues  related  to 
the  budget  deficit,  health  care  alternatives,  the 
environment,  and  novel  applications  of  military 
forces  in  addition  to  traditional  military  topics. 
(Survey  respondents  also  gave  several  '"write- 
in"  responses,  with  most  such  threats  relating  to 
the  use  of  chemical  or  biological  weapons.) 


ORGANIZATIONS 

The  survey  asked  the  individual  respon¬ 
dents  a  number  of  questions  about  their  analyti¬ 
cal  organizations.  By  virtue  of  the  fact  that 
respondents  were  individuals,  rather  than  orga¬ 
nizations,  the  survey  is  not  likely  to  reflect  the 
proportions  of  organizations  exhibiting  the  char¬ 
acteristics  expressed  by  the  respondents.  Rather, 
the  results  represent  the  proportions  of  the  in¬ 
dividual  respondents  who  have  certain 
perceptions  about  their  organizations. 

Most  of  the  respondents  (63%)  believed  that 
recent  events  had  prompted  their  organizations 
to  change.  A  similar  proportion  (65%)  felt  that 


Threat 


Figure  10:  Potentials  Threats 
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decision-makers  were  asking  more  of  them  in 
1993  than  during  the  1983-1987  period.  Despite  a 
large  number  of  individuals  perceiving  change 
in  their  organizations,  that  change  was  not 
uniformly  evidenced  in  terms  of  staff  reductions. 

Figure  11  shows  the  relative  proportions  of 
individuals  belonging  to  organizations  that  had 
changed  the  sizes  of  their  staffs.  In  government 
circles,  a  greater  proportion  of  Army  respon¬ 
dents  had  seen  their  organizations  downsized 
(military  68%,  civil  service  77%).  The  Air  Force 
(military  60%,  civil  service  57%)  and  the  Navy 
(military  36%,  civil  service  46%)  follow. 
Significantly  fewer  individuals  in  private 
organizations  (39%)  had  experienced  such 
down-sizing. 


Figure  11:  Right-Sizing  Profile  by 
Organization 


SUMMARY 

The  survey's  intent  was  to  provide  insight 
on  the  MOR  community's  response  to  change. 
When  asked  directly,  66%  of  the  survey  respon¬ 
dents  thought  the  military  analytic  community 
was  indeed  responding  to  changes.  Furthermore, 
we  believe  that  the  survey  responses  indicate 
that  the  MOR  community  has  the  intrinsic 
capability  to  be  responsive. 

The  survey  respondents  characterized  the 
MOR  community  as  well  experienced,  highly 
educated,  and  interdisciplinary.  We  perceive  mil¬ 
itary  OR  community  as  containing  a  healthy 


distribution  of  experience,  with  a  good  mix  of 
newcomers,  who  tend  to  question  dogma  and 
time-honored  approaches,  and  senior  members, 
who  provide  the  insights  learned  from  their 
experience  and  corporate  memory. 

The  community  is  also  remarkably  well 
educated  and,  apparently,  eager  to  continue  to 
learn.  Even  with  84  percent  of  the  respondents 
having  earned  advanced  degrees,  18  percent 
were  enrolled  in  university  courses,  and  60% 
were  involved  in  self-study.  The  community  has 
also  retained  its  original  interdisciplinary  nature. 
It  is  well-rounded  in  terms  of  professional  and 
organizational  affiliations,  providing  multiple 
perspectives  on  issues  of  national  importance. 

Clearly,  the  military  OR  community  has  the 
ingredients:  experience,  education,  and  an  inter¬ 
disciplinary  focus:  to  effectively  respond  to 
change.  The  real  issue  is  whether  or  not  the  com¬ 
munity  is  living  up  to  that  potential.  The  survey 
provided  only  a  few  clues  to  the  community's 
actual  adaptiveness.  The  shift  in  primary  the¬ 
aters  of  interest  indicates  a  responsiveness  to 
changing  world  conditions,  and  the  change  in 
primary  areas  of  interest  reflects  a  growing 
emphasis  on  the  effectiveness  of  integrated 
forces  at  the  joint  and  theater  levels.  Even  the 
ranking  of  the  most  significant  threats  to  nation¬ 
al  security  reflects  the  respondents  recognition  of 
a  drastically  different  socio-political  environ¬ 
ment.  These  indicators  imply  that  the  MOR  com¬ 
munity  is,  indeed,  responding  to  change.  The 
survey,  however,  did  not  address  the  ultimate 
effectiveness  of  MOR  in  responding  to  change. 

Military  Operations  Research  is  most  useful 
when  it  is  an  integral  component  of  the  decision 
process.  Is  the  MOR  community  illuminating 
issues,  defining  alternatives,  and  focusing  atten¬ 
tion  on  the  right  measures  so  that  the  military 
commanders  and  national  policy  decision  mak¬ 
ers  can  make  the  best  decisions  possible  with  the 
available  data?  That  question  can  be  answered 
effectively  only  by  the  consumers  of  MOR  analy¬ 
sis.  If  the  community  continues  to  build  on  its 
heritage  of  satisfying  the  problem-solving  needs 
of  its  customers,  MOR  will  do  more  than  simply 
respond  to  change.  It  will  be  in  strong  demand 
and  have  a  major  role  in  defining  the  changes  of 
the  future.  Simply  put,  our  future  is  up  to  us. 
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APPENDIX 

The  following  is  a  summary  of  the  questions 
posed  to  the  MORS  membership  in  the  survey. 

Who  Are  We? 

la.  How  many  years  have  you  been  an 
analyst? 

lb.  Of  these,  have  many  have  focused  on 
MOR? 

lc.  How  much  longer  do  you  anticipate  being 
active  in  military  analysis? 

2.  What  is  your  affiliation  with  the  military? 

•  US  Army  (military  or  civil  service?) 

•  US  Navy  (military  or  civil  service?) 

•  US  Air  Force  (military  or  civil  service?) 

•  Joint  Staff  (military  or  civil  service?) 

•  OSD  (military  or  civil  service?) 

•  FFRDC 

•  Professional  Services  Firm 

•  Manufacturing  Firm 

•  Academic 

•  Consultant 

•  Other 

3a.  Education-Highest  Level  Attained: 

•  High  School 

•  Bachelors 

•  Bachelors  Plus 

•  Masters 

•  Masters  Plus 

•  Doctorate 

3b.  Gender  (male  or  female) 

3c.  How  do  you  describe  your  job  to  people 
unfamiliar  with  analysis? 

•  Engineer 

•  Scientist 

•  Mathematician/Statistician 

•  Computer  Specialist 

•  Consultant 

•  Industrial  Engineer 

•  Operator  (pilot,  infantry,  etc.) 


•  Systems  Analyst 

•  Other 

What  is  Our  Focus? 

4.  Indicate  your  primary  area  of  analysis  in 
each  of  the  time  periods  1983-87,  1988-92, 
and  1993-97. 

•  System  or  Individual  Performance 

•  Combined  Arms  or  Unit  Effectiveness 

•  Conventional  Campaign  Contribution 

•  Strategic  Campaign  Contribution 

•  System  Cost 

•  Force  Structure  Cost 

•  Test  &  Evaluation 

•  Manpower  or  Force  Management 

•  Operations  &  Maintenance 

•  Logistics 

•  Readiness 

•  Sustainability 

•  Training 

•  Other 

5.  Indicate  your  primary  theater  of  analysis  in 
each  of  the  time  periods  1983-87,  1988-92, 
and  1993-97. 

•  Central  Europe 

•  NATO  Hank 

•  Eastern  Europe 

•  Southwest  Asia 

•  Eurasia 

•  Northeast  Asia 

•  Southeast  Asia 

•  Pacific  Rim 

•  Africa 

•  American  Hemisphere 

•  Continental  United  States 

•  Space 

•  Not  Theater  Dependent 

6.  Indicate  the  primary  time  frame  consid¬ 
ered  in  your  own  analyses  in  each  of  the 
time  periods  1983-87, 1988-92,  and  1993-97. 

•  Historical  Perspective 

•  Immediate  Application 

•  Less  than  1  year 

•  1  to  5  years 
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•  Greater  than  5  years 

•  No  Particular  Time  Frame 

7.  Indicate  the  primary  factor  that  limits  the 
utility  of  your  analysis  by  the  decision 
maker  in  each  of  the  time  periods  1983-87, 
1988-92,  and  1993-97. 

•  Accuracy 

•  Precision 

•  Presentation 

•  Robustness 

•  Timeliness 

8.  Indicate  your  own  primary  need  today 
within  each  factor  that  limits  the  utility  of 
your  analysis. 

Factors: 

•  Accuracy 

•  Precision 

•  Presentation 

•  Robustness 

•  Timeliness 

Needs: 

•  Scenarios 

•  Accepted  Assumptions 

•  Analytic  Techniques 

•  Data 

•  Software 

•  Hardware 

9.  What  is  your  prognosis  (excellent,  good, 
poor,  bad)  for  you  seeing  significant 
improvements  in  each  of  the  following? 

•  Scenarios 

•  Accepted  Assumptions 

•  Analytic  Techniques 

•  Data 

•  Software 

•  Hardware 

10.  Do  you  need  formal  training  to  learn: 

•  Analytical  Techniques? 

•  Computer  Techniques? 

•  Functional  Knowledge? 
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11 .  If  you  need  more  training: 

•  is  it  available  from  your  organization? 

•  are  you  taking  university  courses? 

•  are  you  working  on  self  study? 

How  Has  the  Threat  Changed? 

12.  Thinking  about  the  next  5  years  or  so, 
which  of  the  following  are  extremely  seri¬ 
ous  threats  to  our  national  security  inter¬ 
ests?  Which  are  the  one  or  two  greatest 
threats? 

•  International  Drug  Trafficking 

•  Possession  of  Nuclear  Weapons  by 
Third  World  Countries  &  Terrorists 

•  Environmental  Problems 

•  Violent  Crime 

•  High  Cost  of  Health  Care 

•  Federal  Budget  Deficit 

•  Economic  Competition  from  Japan  and 
Europe 

•  Terrorist  Activities  around  the  World 

•  Manmade  Changes  in  Global  Climate 

•  Radiation  from  Old  Nuclear  Reactors  & 
Nuclear  Waste 

•  Armed  Conflicts  in  the  Middle  East 

•  Armed  Conflicts  in  Central  America 

•  Russian  Nuclear  Weapons 

•  Russian  Aggression  Around  the  World 

Have  We  Changed? 

13.  In  your  opinion,  is  the  military  analytic 
community  responding  to  the  changes 
from  the  last  few  years? 

14.  Are  decision  makers  asking  more  or  less  of 
you  than  during  the  1993-87  time  period? 

15.  Have  the  changes  in  recent  years  caused 
your  local  analysis  community  to 
reorganize? 

16.  If  so,  are  you  more  or  less  centralized  as  a 
separate  department  with  an  analysis 
identity? 

17.  Is  your  organization  getting  larger  or 
smaller? 
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MORS  offers  two  prizes  for  best  papers — the  Barchi  Prize  and  the  Rist  Prize.  The  Rist  Prize  will  be  awarded  to 
the  best  paper  in  military  operations  research  submitted  in  response  to  this  Call  for  Papers.  The  Barchi  Prize  will 
be  awarded  to  the  best  paper  from  the  entire  65th  symposium,  including  Working  Groups,  Composite  Groups, 
and  General  Sessions. 

David  Rist  Prize:  Papers  submitted  in  response  to  this  call  will  be  eligible  for  consideration  for  the  Rist  Prize. 
The  committee  will  select  the  prize-winning  paper  from  those  submitted  and  award  the  prize  at  the  66th 
MORSS.  If  selected,  the  author(s)  will  be  invited  to  present  the  paper  at  the  66th  MORSS  and  to  prepare  it  for 
publication  in  the  MORS  journal.  Military  Operations  Research.  The  cash  prize  is  $1000.  To  be  considered,  the 
paper  must  be  mailed  to  the  MORS  office  and  postmarked  no  later  than  September  30th,  1997.  Please  send  the 
original,  three  copies  and  the  disk. 

Richard  H.  Barchi  Prize:  Author(s)  of  those  papers  selected  as  the  best  from  their  respective  Working  Group 
or  Composite  Group,  and  those  of  the  General  Sessions  at  the  65th  MORSS  will  be  invited  to  submit  the  paper 
for  consideration  for  the  Barchi  Prize.  The  committee  will  select  the  prize-winning  paper  from  among  those 
presented,  nominated  and  submitted.  The  prize  will  be  presented  at  the  66th  MORSS.  The  cash  prize  is  $1000. 
To  be  considered,  the  paper  must  be  mailed  to  the  MORS  office  and  postmarked  no  later  than  November  28, 
1997.  Please  send  the  original,  three  copies  and  a  disk. 

Prize  Criteria 

The  criteria  for  selection  for  both  prizes  are  valuable  guidelines  for  presentation  and/or  submission  of  any 
MORS  paper.  To  be  eligible  for  either  award,  a  paper  must,  at  a  minimum: 

•  Be  original  and  a  self-contained  contribution  to  systems  analysis  or  operations  research; 

•  Demonstrate  an  application  of  analysis  or  methodology,  either  actual  or  prospective; 

•  Prove  recognizable  new  insight  into  the  problem  or  its  solution;  and 

•  Not  previously  been  awarded  either  the  Rist  Prize  or  the  Barchi  Prize  (the  same  paper  may  compete  for  but 
cannot  win  both  prizes.) 

Eligible  papers  are  judged  according  to  the  following  criteria: 

Professional  Quality 

•  Problem  definition 

•  Citation  of  related  work 

•  Description  of  approach 

•  Statement  of  assumptions 

•  Explanation  of  methodology 

Contribution  to  Military  Operations  Research 

•  Importance  of  problem 

•  Contribution  to  insight  or  solution  of  the  problem 

•  Power  of  generality  of  the  result 

•  Originality  and  innovation 


•  Analysis  of  data  and  sources 

•  Sensitivity  of  analyses  (where  appropriate) 

•  Logical  development  of  analysis  and  conclusions 

•  Summary  of  presentation  and  results 


ABSTRACT 

This  paper  reports  on  a  prototype  per¬ 
formance  evaluation  or  benchmarking 
approach  for  Department  of  Defense  (DoD) 
business-like  operations  using  data  envelop¬ 
ment  analysis  (DEA).  These  activities  do  not 
have  a  pure  profit  or  'l^ottom-line"  indicator 
of  performance,  and  therefore,  performance 
must  be  evaluated  by  some  other  means. 
The  study  illustrates  a  methodology  for 
identifying  best  practices,  providing  bench¬ 
marks,  quantifying  performance  gaps,  and 
improving  management  control  and  effi¬ 
ciency  for  these  types  of  operations  via  using 
data  for  the  DoD  commissary  system. 
Although  this  study  focuses  on  DoD 
commissaries,  the  technique  is  equally 
applicable  to  other  government  activities. 

I.  INTRODUCTION 

The  Air  Force,  Army,  Navy,  Marine 
Corps,  and  other  Department  of  Defense 
(DoD)  agencies  operate  several  business-like 
activities  for  the  benefit,  morale,  welfare, 
and  recreation  of  military  members  and 
their  dependents.  Examples  of  these  activi¬ 
ties  are  commissaries  (grocery  stores), 
exchanges  (department  stores),  noncommis¬ 
sioned  officers  clubs  (restaurants),  child 
development  centers,  golf  courses,  and 
bowling  centers.  These  activities  generate 
revenues  of  over  $15  billion  and  are  referred 
to  as  business-like  since  they  provide  ser¬ 
vices  similar  to  many  commercial  busi¬ 
nesses.  In  addition,  they  are  similar  to 
businesses  in  that  they  need  to  continuously 
improve  and  operate  efficiently  if  they  are  to 
be  successful  in  their  endeavors.  However, 
they  differ  from  commercial  enterprises  in 
that  they  are  primarily  intended  to  provide  a 
service  to  the  military  member,  do  not  have 
a  pure  profit  motive,  and  their  operating 
revenues  (e.g.,  patron  fees,  dues,  and  sales) 
are  subsidized  by  Congress  through 
appropriations. 

In  recent  years,  total  quality  manage¬ 
ment  (TQM)  has  been  infused  into  the  oper¬ 
ations  of  many  businesses  and  government 
operations.  A  key  component  of  TQM  is 
benchmarking.  Benchmarking  is  the  identifi¬ 
cation  of  best  practices  leading  to  contin¬ 
uous  improvement  and  more  efficient 
operations. 

The  purpose  of  this  study  is  to  demon¬ 
strate  the  applicability  of  DEA  as  a 


benchmarking  tool  to  aid  DoD  business-like 
activities  in  identifying  superior  perfor¬ 
mance,  quantifying  performance  gaps,  and 
other  benchmarking  considerations.  Com¬ 
missaries  are  selected  as  the  organization  to 
be  evaluated  because  of  the  intense  interest 
by  Congress  and  others  in  reducing  the 
appropriated  fund  support  for  commissaries 
and  in  making  them  more  efficient.  The 
analysis  identifies,  to  management  and 
others,  areas  that  may  be  most  suitable  for 
commissary  improvement.  Thus,  this  in¬ 
formation  becomes  the  catalyst  for  change 
and  continuous  improvement  required  by 
TQM  concepts. 

In  recent  years,  there  have  been  several 
studies  on  the  Military  Services'  commissary 
operations.  Generally,  the  genesis  of  these 
studies  has  been  concern  over  the  amount  of 
appropriated  funds  used  to  support  the 
commissary  systems  and  the  desire  to  find  a 
more  efficient  and  effective  way  of  securing 
the  commissary  benefit  for  the  military 
member.  The  President's  Commission  on 
Privatization  [1988]  recommended  that  the 
private-sector  manage  and  operate  the  mili¬ 
tary  commissaries.  The  Jones  Commission 
study  [1989]  resulted  in  the  consolidation  of 
the  individual  Service's  commissary  systems 
into  the  Defense  Commissary  Agency 
(DEC A)  in  1991.  Finally,  a  project,  Unit  Cost 
Resourcing,  has  been  initiated  to  improve 
productivity  by  measuring  productivity 
with  a  single  input  to  single  output  ratio — 
total  appropriated  dollar  cost  per  dollar 
sales  [Shycoff,  1990]. 

Following  this  introduction,  the  paper  is 
organized  in  the  following  manner.  Next  is  a 
section  briefly  discussing  benchmarking  and 
describing  the  methodology  used  in  this 
research  to  measure  commissary  perfor¬ 
mance,  i.e.,  data  envelopment  analysis 
(DEA).  This  is  followed  by  a  section  which 
describes  the  data  used  in  this  study.  Finally, 
there  is  a  discussion  of  the  results  of  the 
analysis,  including  insights  into  manage¬ 
ment  control  of  commissary  operations  and 
possible  benchmarks  available  from  the 
analysis. 

II.  METHODOLOGY 
Benchmarking 

Benchmarking  refers  to  the  practice  of 
searching  for  and  identifying  the  best 
practices  of  a  particular  class  or  type  of 
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organization.  These  "best  operations"  then 
become  the  benchmark  by  which  the  perfor¬ 
mance  of  similar  activities  is  evaluated.  The 
focus  is  on  comparing  an  organization's  current 
performance  with  the  very  best  that  has  actually 
been  achieved. 

Once  the  activities  with  the  best  perfor¬ 
mance  are  identified,  they  can  be  analyzed  to 
assist  in  identifying  techniques,  processes,  and 
procedures  that  can  be  used  to  increase  the  per¬ 
formance,  e.g.,  effectiveness  or  efficiency,  of  a 
similar  activity.  The  source  of  these  referenced 
best-practices  organizations  may  be  internal,  i.e., 
from  another  part  of  the  company,  or  external, 
i.e.,  from  a  competitor.  Each  source  has  its  own 
advantages  and  limitations.  Internal  compar¬ 
isons  are  used  in  this  study  for  the  reasons 
outlined  below. 

Spendolini  (1992,  p.  16)  and  Camp  (1989,  p. 
61)  recognize  that  internal  benchmarking  is  a 
good,  first-step  in  benchmarking  investigations. 
Before  starting  an  external  benchmarking  study, 
it  is  important  that  an  organization  thoroughly 
understand  its  own  operations  and  identifies  the 
critical  issues  that  will  be  faced  during  an  exter¬ 
nal  investigation.  This  is  accomplished  with  an 
internal  benchmarking  approach. 

An  internal  comparison  offers  three  addi¬ 
tional  advantages:  (1)  the  relevant  data  needed 
for  benchmarking  are  normally  readily  available 
or  can  be  made  available,  with  the  right  manage¬ 
ment  emphasis,  to  initiate  such  a  study,  (2)  data 
elements  are  consistently  defined,  and  (3)  man¬ 
agement  more  readily  accepts  the  results  because 
they  accept  the  comparability  of  the  benchmark 
partners  (the  comparison  organizations). 

On  the  other  hand,  limitations  of  an  internal 
comparison  include  that  it  runs'the  risk  of  identi¬ 
fying  the  best  practices  of  a  poorly  run  organiza¬ 
tion  and  not  taking  advantage  of  new  ideas 
generated  by  others.  To  compensate  for  this 
problem,  a  comparison  to  competitors  or  to 
other  external  organizations  may  be  desirable. 
However,  the  relevant  data  can  be  difficult,  if  not 
impossible,  to  obtain,  data  elements  may  not  be 
consistently  defined,  and  acceptance  by  manage¬ 
ment  may  be  constrained  because  of  the  belief 
that  "they  aren't  like  us."  An  attempt  was  made 
to  obtain  comparable  data  from  commercial  gro¬ 
cery  chains  so  that  external  benchmarking  could 
be  done  for  this  study  but  the  companies  were 
not  interested  in  participating. 


Using  data  envelopment  analysis,  we  iden¬ 
tify  "best-in-class"  performance  through  internal 
benchmarking.  In  addition  to  identifying  the 
best  performing  commissaries,  DEA  also  pro¬ 
vides  some  insights  into  why  those  commis¬ 
saries  are  the  best  performers.  As  described  in 
the  next  section,  it  identifies  performance  gaps  in 
the  input  variables  for  each  poorer  performing 
commissary.  DEA  also  identifies  the  superior 
performing  commissaries  to  which  the  managers 
of  poorer  performing  commissaries  should  turn 
to  for  assistance  in  identifying  their  operational 
shortcomings,  e.g.,  identifying  enablers. 

Data  Envelopment  Analysis 

Data  envelopment  analysis  was  developed 
by  Charnes,  Cooper,  and  Rhodes  (CCR)  [1978] 
for  measuring  efficiency  of  not-for-profit  entities. 
Since  its  inception,  it  has  been  further  developed, 
extended,  and  applied  in  a  variety  of  ways  and 
settings  [Banker  et.  al,  1989].  A  good  summary  of 
the  technical  computational  details  of  the 
methodology  and  additional  description  of  the 
various  DEA  models  can  be  found  in  several  ref¬ 
erences  including  Callen  [1991]  and  Banker 
et.  al  [1989]. 

There  are  several  variations  of  the  DEA 
model.  Here  we  take  a  closer  look  at  the  CCR 
fractional  programming  version  of  the  DEA 
model  because  it  more  clearly  provides  insights 
into  the  characterizations  of  the  DEA  model  and 
the  reasons  for  its  appropriateness  as  a  bench¬ 
marking  tool.  This  model  is  as  follows: 
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where: 

=  The  performance  rating  sought  for  organi¬ 
zation  "o"  which  is  the  one  member  of  the 
reference  set,/=l,...,n,  that  is  to  be  evaluated 
relative  to  the  others. 

Vi  =  The  variable  weights  for  each  type  of  input, 
which  will  be  determined  by  the 
solution  of  the  model. 

Uj.  =  The  variable  weights  for  each  type  of  out¬ 
put,  r=l,...,s,  which  will  be  determined  by 
solving  the  model. 

—  The  known  amount  of  input  "i"  used  by 
organization  "'o"  during  the  period  of 
observation. 

=  The  known  amount  of  output  "r"  pro¬ 
duced  by  organization  "o"  during  the 
period  of  observation. 

x-  =  The  known  amount  of  input  "i"  used  by 
organization  during  the  period  of 
observation. 

=  The  known  amount  of  output  "r"  produced 
by  organization  "j"  during  the  period  of 
observation. 

e  =  A  positive  constant  introduced  to  insure  that 
all  of  the  observed  inputs  and  outputs  will 
have  some  positive  value  assigned  to  them. 

In  the  CCR  ratio  form,  the  problem  is  inter¬ 
preted  as  one  of  choosing  and  z;-  values,  sub¬ 
ject  to  the  constraints,  that  will  assign  the 
maximum  possible  ratio  value,  which  is  to  be 
used  as  the  performance  rating  of  organization 
"o".  The  essential  aspect  of  the  CCR  ratio  con¬ 
struction  is  the  reduction  of  the  multiple  output, 
multiple  input  situation  for  each  organization  to 
that  of  a  single  "virtuaF  output  and  a  single 
"virtual"  input  for  each  organization.  Formula 
(l)'s  objective  then  is  to  maximize  the  ratio 
of  virtual  output  to  virtual  input  for  each 
organization  in  the  population  under  evaluation. 

The  multipliers  or  weights,  u  and  v,  are 
determined  by  the  model  directly  from  the  data 
and  not  assigned  a  priori.  A  'laest"  set  of  weights 


is  determined  for  each  of  the  organizations  to  be 
evaluated.  Given  this  best  set  of  weights,  each 
organization  "o"  is  evaluated  on  whether  any 
other  organization  "j"  achieved  a  higher  ratio 
than  organization  "o"  using  the  latter's  best 
weights.  DBA  thus  identifies  the  subset  of  orga¬ 
nizations  that  have  the  best  virtual  output  to  vir¬ 
tual  input  ratio  and  thereby,  determines  a 
performance  frontier  based  on  those  "best" 
performing  organizations. 


Figure  1:  DBA  Graphical  Representation 


Figure  1  is  a  simplified  graphical  representa¬ 
tion  of  DBA.  Y  is  a  vector  of  multiple  outputs 
which  become  the  virtual  output  and  X  is  a  vec¬ 
tor  of  multiple  inputs  which  become  the  virtual 
input.  DBA  creates  a  frontier  (curve  A)  based  on 
observed  data  for  the  "best"  organizations,  i.e., 
the  organizations  with  the  best  ratio  of  multiple 
outputs  (the  virtual  output)  to  multiple  inputs 
(the  virtual  input).  These  frontier  organizations 
(e.g.,  point  B)  are  best  performers,  receive  a  rat¬ 
ing  of  one,  and  are  used  to  assess  the  perfor¬ 
mance  of  other  similar,  but  less  productive 
operations.  The  less  productive  organizations 
(e.g.,  points  C  and  D)  receive  a  rating  of  less  than 
one  depending  on  their  distance  from  the 
frontier. 

DBA  has  several  characteristics,  described  in 
the  following  paragraphs,  which  make  it  suited 
as  a  benchmarking  tool.  First,  DBA  is  empirically 
based.  That  is,  the  solution  to  the  model  is  based 
on  observed  data  and  behavior.  DBA  does  not 
require  a  priori  identification  of  the  underlying 
functional  form  to  relate  the  outputs  and  inputs 
nor  the  "weights"  (u  and  v)  to  indicate  the 
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relative  importance  of  each  output  and  input 
used  to  determine  the  solution  to  the  model. 
Thus,  the  solution  to  the  model  is  not  based  on 
some  hypothetical  or  theoretical  construct  but  on 
actual  performance  and  must  conform  to  the 
actual  performance  of  an  entity.  This  makes  the 
results  of  the  model  more  acceptable  to  man¬ 
agers  which  is  an  important  consideration  for  a 
benchmarking  model.  In  addition,  not  requiring 
a  priori  identification  of  a  production  function  is 
particularly  advantageous  for  this  study  since  an 
appropriate  production  function  has  not  yet 
been  developed  for  commissaries. 

To  get  a  better  understanding  of  this  charac¬ 
teristic,  it  is  sometimes  helpful  to  compare  DEA 
to  index  number  and  regression  approaches. 
Because  of  the  absence  of  any  need  for  prescrib¬ 
ing  the  underlying  functional  form,  weights  etc., 
in  an  a  priori  manner,  DEA  contrasts  with  in¬ 
dexes  or  regression  approaches  which  are 
dependent  on  numerous  assumptions  and,  in 
principle,  require  a  great  deal  of  analytical  theo¬ 
rizing  prior  to  choosing  the  form(s)  in  which 
they  are  used.  However,  in  reality,  the  actual, 
observed  forms  are  variable  in  nature — possibly 
differing  for  each  organization  under  evaluation. 
Consequently,  with  DEA  more  flexibility  is  per¬ 
mitted  in  the  form  of  the  actual  production  func¬ 
tion  employed  by  each  organization  under 
evaluation  than  is  true  with  regression  and  index 
number  approaches. 

Second,  DEA  is  an  extremal  methodology 
which  optimizes  on  each  observation  to  estimate 
a  performance  frontier  by  coming  as  close  as 
possible  to  each  observation.  That  is,  DEA 
employs  "n"  optimizations,  one  for  each  entity 
being  evaluated.  Consequently,  DEA  not  only 
distinguishes  between  best  performing  and  infe¬ 
rior  or  average  performing  operations,  it  also 
provides  information  on  what  weaker  opera¬ 
tions  can  do  to  move  to  higher  levels  of  perfor¬ 
mance.  The  best  performing  organizations 
become  the  benchmark  (comparison  set  of  orga¬ 
nizations)  for  judging  the  performance  of  the 
other  organizations  and  provide  the  basis  for 
DEA  to  determine  the  levels  of  output  and  input 
the  unit  under  evaluation  should  achieve  to  be 
on  the  performance  frontier.  These  efficient  lev¬ 
els  of  output  and  input  are  the  goals  for  this 
organization  developed  from  best-practices 
organizations.  In  benchmarking  terminology. 


the  difference  between  the  DEA  computed 
performance  and  the  actual  performance 
observed  is  the  performance  gap. 

Note  also  that  DEA's  frontier  orientation  dif¬ 
fers  from  the  interior  or  average  orientation 
embodied  in  least-squares  regression.  Least- 
squares  regression  analysis  uses  a  single  opti¬ 
mization  to  come  as  close  as  possible  to  all  of 
these  points.  Thus,  least-squares  regression  tends 
towards  determining  an  average  performance 
and  does  not  make  any  separations  between 
good  and  bad  performance. 

Third,  DEA  handles  multiple  output  and 
multiple  inputs  simultaneously  and  in  a  manner 
that  recognizes  the  simultaneous  variations  and 
interactions  between  the  many  components. 
Thus,  the  resulting  rating  is  a  type  of  total-factor 
productivity  measure.  A  total-factor  productivity 
measure  is  desired  since  single  or  partial  mea¬ 
sures,  such  as  output  per  unit  of  labor  input 
(labor  productivity)  or  output  per  unit  of  capital 
input  (capital  productivity),  seldom  capture  the 
multiple  interests,  goals,  or  values  of  the  entity 
being  evaluated.  Therefore,  these  alternative 
measures  may  provide  incomplete  and  mislead¬ 
ing  indications  of  performance.  This  is  an  appar¬ 
ent  limitation  of  the  single  measure  used  in  the 
Unit  Cost  Resourcing  project  referenced  above. 

Fourth,  as  shown  in  Ali  and  Seiford  [1990], 
because  of  DEA's  optimization  process,  the  units 
of  measure  for  the  input  and  output  variables  do 
not  influence  the  performance  measure  com¬ 
puted  as  long  as  the  unit  of  measure  for  each 
variable  is  the  same  for  all  organizations. 
Therefore,  the  variables  can  be  measured  in  dol¬ 
lar  or  nondollar  units.  Additionally,  even  if  all 
variables  are  measured  in  the  same  units  (e.g., 
dollars),  the  variables  still  maintain  their 
individual  identities. 

Finally,  the  aggregate  scalar  efficiency  rating 
computed  by  the  CCR  DEA  model  can  be  fur¬ 
ther  separated  into  scalar  measures  of  technical 
and  scale  efficiencies  [Banker,  1984  and  Banker 
et.  al.,  1984].  Technical  efficiency  is  attained  only 
if  the  outputs  cannot  be  increased  or  inputs 
decreased  without  decreasing  some  output  or 
increasing  some  input.  In  other  words,  technical 
efficiency  exists  when  output  is  the  greatest 
amount  possible  for  the  input  amounts  con¬ 
sumed.  Scale  efficiency  exists  when  an  organiza¬ 
tion  is  producing  at  the  optimum  scale,  i.e.,  a 
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change  in  the  inputs  results  in  a  proportionate 
change  in  the  output  (constant  returns  to  scale). 

The  above  discussion  describes  the  attrib¬ 
utes  that  make  DEA  attractive  and  suitable  for 
this  particular  research.  However,  there  are 
model  limitations  that  should  be  kept  in  mind. 
First,  DEA  requires  that  all  inputs  and  outputs 
have  positive  values.  Thus,  if  a  variable  has  a 
zero  or  negative  value,  it  must  be  adjusted. 
There  are  techniques  for  doing  this  and,  gener¬ 
ally,  this  will  not  cause  a  problem  in  the  DEA 
evaluation  because  the  model  is  unaffected  by 
the  units  of  measure  as  long  as  they  are  consis¬ 
tent  for  each  variable  across  all  organizations 
included  in  the  evaluation  set. 

Second,  the  data  must  consist  of  a  relatively 
homogeneous  set  of  organizations,  and  there 
needs  to  be  three  observations  (organizations) 
for  each  variable  used  in  the  model  in  order  to 
have  sufficient  degrees  of  freedom  for  a  valid 
evaluation.  With  less  than  three  observations  per 
variable,  the  model  will  tend  to  classify  or  rate 
each  organization  as  performing  100%  efficient 
because  it  will  heavily  weight  the  one  variable 
that  is  most  advantageous  to  each  organization. 

Finally,  because  of  DEA' s  optimizing  princi¬ 
ple,  it  cannot  be  used,  as  regression  can,  for  gen¬ 
eral  characterizations  of  the  data  nor  for 
prediction  of  future  behavior  of  the  entire  collec¬ 
tion  of  observations.  Additionally,  since  there  is 
no  random  error  term,  the  model  does  not 
directly  consider  that  the  performance  (i.e.,  out¬ 
put  attained  or  input  consumed)  may  be  an 
anomaly  and  not  representative  of  normal  oper¬ 
ations.  Furthermore,  no  statistical  test  of 
significance  can  be  applied  to  the  "weights." 

IV.  DATA 

Outputs  and  Inputs 

In  this  study,  we  use  data  that  were  being 
reported  by  each  of  the  military  services  to  the 
Department  of  Defense — one  output,  sales  dol¬ 
lars,  and  three  inputs,  personnel  work-years, 
store  square  footage,  and  appropriated  fund 
expenditures.  For  a  preliminary  study  such  as 
this  one,  it  is  desirous  and  advantageous  to  use 
data  that  are  understood  and  accepted  by  man¬ 
agement  so  that  management  can  more  readily 
relate  to  and  understand  the  results.  Thus,  since 


these  data  are  being  collected,  reported,  and 
used,  management  understands  them  and  has 
accepted  them  as  being  informative  and  relevant 
in  evaluating  the  performance  of  the  organiza¬ 
tion.  The  input  and  output  data  are  limited  but 
sufficient  to  capture  a  significant  portion  of 
efforts  and  accomplishments  of  commissaries. 

Sales,  Sales  is  one  measure  of  a  commissary's 
accomplishments.  It  is  what  the  commissaries 
are  attempting  to  produce  and  provides  an  over¬ 
all  indication  of  how  well  the  commissary  is 
doing  to  include  partially  measuring  how  well 
individual  commissaries  are  satisfying  its  cus¬ 
tomers.  It  is  an  aggregate  measure  that  repre¬ 
sents  the  customer's  sense  of  the  balance 
between  cost  and  value  received.  The  greater  the 
level  of  sales  (for  a  given  level  of  input),  the  bet¬ 
ter  and  more  effective  the  commissary  is  in  satis¬ 
fying  its  customers.  Since  sales  level  (in  dollars) 
is  the  only  output  measure  currently  being 
reported  by  the  military  commissary  system,  it  is 
the  only  output  used  in  this  study. 

Work-years  (WY).  This  input  measures  the 
amount  of  labor  used  in  producing  the  sales.  The 
greater  the  number  of  work-years,  the  greater 
the  amount  of  sales  we  would  expect.  It  includes 
both  part-time  and  full-time  civilian  and  military 
employees.  The  part-time  work-years  are 
converted  to  full-time  equivalents. 

Square  Feet  (SQFT).  This  input  variable  is  a 
surrogate  for  the  capital  (equipment  and  facili¬ 
ties)  used  in  producing  the  sales.  The  greater  the 
square  footage,  the  larger  the  investment  in  the 
facility  and  equipment  used  in  producing  the 
sales.  It  is  recognized  that  there  are  limitations  in 
using  this  measure  as  a  surrogate  for  the  capital 
investment,  e.g.,  it  does  not  reflect  the  age  of  the 
equipment  or  facilities.  However,  it  does  capture 
the  size  aspect  of  the  capital  investment.  In  addi¬ 
tion,  although  this  variable  is  not  directly  con¬ 
trollable  by  local  management,  it  does  provide 
useful  information  relative  to  how  well  a  fixed 
asset  is  being  used  and  the  appropriate  scale  of 
operations  that  is  required  for  the  particular 
market  in  which  that  commissary  is  operating. 

Appropriated  Funds  (APF).  APF  are  the  funds 
provided  by  Congress  to  partially  support  the 
operation  of  commissaries.  The  amount  of  APF 
used  in  support  of  commissaries  is  a  key  concern 
of  Congress  and  the  military  services.  Efficient 
use  of  these  funds  is  necessary  to  insure 
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continued  Congressional  support  for  and  main¬ 
tenance  of  this  military-service  benefit.  The 
expected  relationship  between  sales  and  APF 
would  be  the  greater  the  amount  of  APF 
support,  the  greater  the  level  of  sales. 

Substituting  the  variables  into  the  DEA 
model  from  (1),  we  have: 

(2)  maximize  = 


^unj^o  +  ^s,f,SQPT,  +  V^^fAPF„ 

uS-, 

subject  to: - 1- 


^un,V^2  +  ^sc,f,SQPT^  +  \f^Pp2 

_ ^5^237 _ 

^ujy ^237  ^sqft^Q^'^237  %/^^^237 


for  commercial  retail  operations  such  as  sales  per 
square  foot  and  sales  per  employee  (per  work- 
year  in  this  case)  along  with  a  measure,  sales  per 
APF  dollar,  that  is  unique  to  the  commissary 
operating  environment. 

Decision  Making  Units 

Charnes,  Cooper,  and  Rhodes  [1978]  define  a 
decision  making  unit  (DMU)  as  an  organization 
or  entity  that  has  the  authority  and  responsibility 
for  managerial  decision  making.  For  this  study, 
DMUs  are  the  stores  operated  by  the  Army,  Air 
Force,  Navy,  and  Marine  Corps  in  the  continen¬ 
tal  United  States  plus  Hawaii  and  Alaska. 
Commissaries  operated  in  Europe,  Japan,  and 
other  overseas  areas  are  excluded  due  to  having 
significantly  different  operating  environments. 
There  are  89  Air  Force  commissaries,  72  Army 
commissaries,  62  Navy  commissaries,  and  14 
Marine  Corps  commissaries,  for  a  total  of  237 
DMUs,  analyzed  in  this  study  over  a  two-year 
period. 

Data  Collection 


^unj^a  +  +  V,i,fAPF^ 


^^o,J^o  +  ^s,ftSQFT„  +  v^pfAPF^ 

where  the  symbols  are  as  previously  defined. 

As  can  be  noted,  these  variables  capture  and 
aggregate  two  commonly  used  measures  of  pro¬ 
ductivity:  labor  productivity,  sales  per  work- 
year,  and  capital  productivity,  sales  per  square 
foot.  Hence,  although  the  data  are  limited,  they 
are  sufficient  to  provide  us  with  ratios  of  the 
type  commonly  used  to  evaluate  performance 


Fiscal  years  1988  and  1989  data  are  used  in 
this  study,  thereby  allowing  both  time-series  and 
cross-sectional  analyses  to  be  completed.  The 
data  are  obtained  from  the  DoD  Commissary 
Operations  Report  (DD-FM&P(A)  1187)  for  each 
year.  This  report  was  prepared  by  each  military 
service's  commissary  service  and  submitted  to 
the  Assistant  Secretary  of  Defense  (Force 
Management  and  Personnel). 

Sales  figures  are  standardized  by  inflating 
FY 1988  sales  to  FY 1989  levels  using  the  produc¬ 
ers  price  index.  FY  1988  APF  amounts  are  nor¬ 
malized  to  FY  1989  levels  by  using  standard 
Department  of  Defense  inflation  rates, 

V.  ANALYSIS 

This  research  consists  of  three  primary 
investigations.  The  first  analysis  is  of  individual 
store  performance  and  illustrates  the  type  of 
benchmark  information  available  from  DEA  for 
individual  stores.  Second,  system- wide  perfor¬ 
mance  is  reviewed  arid  the  impact  of  store  size  is 
examined  so  as  to  provide  insight  on  potential 
improvement  system-wide.  The  final  part  is  a 
sensitivity  analysis  assessing  the  impact  of 
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different  variables  on  the  reported  results.  Note 
that  this  is  not  a  complete  benchmarking  study 
because  we  do  not  fully  identify  performance 
''enablers."  The  intent  is  to  stimulate  man¬ 
agement  into  identifying  opportunities  for 
improving  commissary  operations. 

Individual  Store  Performance 

As  previously  discussed,  DBA  provides  a 
summary  measure  of  performance  along  with 
indicating  the  amount  each  input  should  be 
reduced  (i.e.,  the  performance  gap)  in  order  for 
the  commissary  to  be  operating  on  the  perfor¬ 
mance  frontier  and  therefore,  operating  at  the 
same  level  as  the  best-performing  commissaries. 
It  also  identifies  the  organizations  on  which  the 
evaluated  unit's  performance  rating  is  based. 
Table  1  illustrates,  for  Maxwell  Air  Force  Base 
commissary,  the  management  (benchmarking) 
information  available  from  the  data  envelop¬ 
ment  analysis.  Note  that  the  following  conclu¬ 
sions  are  tempered  by  the  need  to  insure  that  the 
variables  used  completely  capture  the  accom¬ 
plishments  and  efforts  of  the  commissary 
system. 

TABLE  1 

MAXWELL  AFB  EFFICIENCY  RESULTS 

1988  1989 

a.  Actual  Variable  Values 

Sales  ($000)  $  25,408  $  26,097 


Work-years 

103 

90.6 

Square  Feet 

30,100 

30,100 

Appropriated  Funds  ($000) 

$  2,092 

$2,175 

b.  Efficiency  Ratings 

Technical  Efficiency 

.612 

.699 

Scale  Efficiency 

.997 

.996 

Aggregate  Efficiency 

.610 

.696 

c.  Input  Variable  Values  If  Efficient  For 

Actual  Sales 

Work-years 

63 

63.3 

Square  Feet 

18,400 

21,000 

Appropriated  Funds  ($000) 

$  1,218 

$  1,519 

d.  Technical  Efficiency — ^Weights  of 

Benchmark  Units 

Fort  Leonard  Wood  (1988) 

.1302 

.2799 

Fort  Belvoir  (1988) 

.2579 

.1417 

Pope  Army  Facility  (1988) 

.5294 

.3919 

Carswell  Air  Force  Base  (1989)  .0825 

.1866 

Total 

1.0000 

1.0000 

Reviewing  section  b  of  Table  1,  Maxwell  is 
relatively  technically  inefficient  with  ratings  of 
only  61%  in  1988  and  nearly  70%  in  1989.  Thus, 
there  is  a  significant  performance  gap  between 
Maxwell  and  the  best-performing,  frontier  com¬ 
missaries  to  which  Maxwell  is  compared.  Given 
Maxwell's  sales  levels  in  1988  and  1989,  DEA 
indicates  that  for  Maxwell  to  operate  efficiently, 
it  needs  to  reduce  its  input  consumption  by  30%- 
39%.  Section  c  of  Table  1  provides  the  input  lev¬ 
els  that  should  have  been  used  to  produce  the 
sales  level  achieved.  Note  that  these  values  are 
the  achievable  performance  goals  that  are 
expected  from  a  benchmarking  process  based  on 
the  superior  performance  of  the  "best"  organiza¬ 
tions  under  review.  For  example,  in  FY  1988, 
Maxwell  should  have  used  only  63  work-years, 
18,400  square  feet,  and  $1,218  thousand  in 
appropriated  funds — compared  to  103  work- 
years,  30,100  square  feet,  and  $2,092  thousand 
(normalized  to  FY  1989  dollars)  APF  actually 
consumed.  The  difference  between  these  superi¬ 
or  performance  levels  and  the  actual  level 
achieved  represent  the  performance  gap  for 
Maxwell. 

Although  some  of  these  findings  may  be  the 
result  of  factors  not  controllable  by  management, 
e.g.,  square  footage,  they  still  indicate  that  per¬ 
formance  gaps  exist  and  input  reductions  are 
possible.  The  exact  amount  of  the  reductions 
necessary  to  bring  this  commissary  in  line  with 
the  best  performing  commissaries  to  which  it 
was  compared  needs  to  be  determined  by  man¬ 
agement  and  may  vary  from  those  provided  by 
DEA  because  of  unique  circumstances  applicable 
to  Maxwell  or  some  anomaly  in  the  operations  of 
the  commissary  for  those  years.  However,  DEA 
does  provide  an  indication  of  the  magnitude  of 
the  reduction,  e.g.,  30%-39%.  In  addition,  even 
though  square  footage  is  not  controllable  by 
management,  DEA  does  indicate  that  the  physi¬ 
cal  plant,  as  represented  by  the  square  footage,  is 
too  large  and  not  being  effectively  used  to 
generate  sales. 

Continuing  the  review  of  section  b  of  Table 
1,  Maxwell  is  nearly  scale  efficient  at  close  to 
100%  for  both  FYs  1988  (.997  or  99.7%)  and  1989 
(.996  or  99,6%).  Thus,  at  its  scale  of  operations,  it 
nearly  gets  a  proportionate  change  in  its  output 
for  a  change  in  its  input,  and  it  is  producing  at  its 
optimum  scale  given  the  particular  technical 
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efficiency  conditions.  Hence,  there  is  little  perfor¬ 
mance  gap  corresponding  to  Maxwell's  scale  of 
operations  and  very  little  improvement  is 
possible  in  its  scale  of  operations. 

Finally,  the  units  listed  in  section  d  of  Table  1 
constitute  the  comparison  set  or  benchmarking 
units  upon  which  Maxwell's  performance  rating 
is  based.  Recall  that  these  comparison  units  are 
on  the  performance  frontier,  are  considered  to  be 
the  best  performing  commissaries,  and  would 
receive  a  DEA  rating  of  one  (1)  or  100%.  In  addi¬ 
tion,  these  commissaries  have  an  output  and 
input  mix  similar  to  Maxwell's  and  should  be 
the  first  to  be  reviewed  for  ideas  for  further 
improvement  in  Maxwell's  performance.  Thus, 
these  are  the  best-practice  activities  with  which 
Maxwell  should  be  initially  compared. 


Note  that  the  value  listed  in  section  d  for 
each  comparison  unit  is  not  its  efficiency  rating. 
These  numbers  represent  the  weight  of  each 
unit's  contribution,  as  objectively  determined  by 
the  DEA  model,  in  determining  Maxwell's  per¬ 
formance  rating.  For  example.  Pope  Army 
Facility  (1988)  had  the  greatest  influence 
on  Maxwell's  1988  rating  with  a  weight  of  .5294 
which  indicates  that  52.94%  of  Maxwell's  rat¬ 
ing  was  based  on  the  Pope  Army  Facility 
performance. 

System-Wide  Performance 

Table  2  summarizes  the  DEA  ratings  by  mili¬ 
tary  service,  type  of  efficiency  (aggregate,  techni¬ 
cal,  or  scale),  and  fiscal  year.  For  example,  we  see 


TABLE  2 

SUMMARY  OF  MEANS  AND  STANDARD  DEVIATIONS  FOR  DEA  PERFORMANCE  RATINGS 


Air  Force 

Armv 

Naw 

Marines 

Aggregate  Efficiency 

1988 

Mean 

.61 

.59 

.52 

.64 

Standard  Deviation 

.13 

.18 

.17 

.12 

1989 

Mean 

.68 

.57 

.53 

.64 

Standard  Deviation 

.15 

.18 

.17 

.12 

Technical  efficiency 

1988 

Mean 

.63 

.64 

.61 

.68 

Standard  Deviation 

,13 

.16 

.13 

.10 

1989 

Mean 

.70 

.61 

.62 

.67 

Standard  Deviation 

.14 

.16 

.12 

.10 

Scale  Efficiency 

1988 

Mean 

.97 

.92 

.85 

.94 

Standard  Deviation 

.06 

.10 

.20 

.05 

1989 

Mean 

.97 

.93 

.85 

.94 

Standard  Deviation 

.05 

.11 

.20 

.06 

Store  Average  Square  Feet 

1988 

22,981 

17,816 

14,107 

14,069 

1989 

23,543 

20,110 

14,168 

14,376 

Page  26 


Military  Operations  Research,  Spring  1996 


BENCHMARKING  AND  EFFICIENT  PROGRAM  DELIVERY 


that  the  mean  aggregate  efficiency  rating  for  Air 
Force  commissaries  in  FY  1988  is  .61  (61%).  The 
Air  Force's  average  technical  efficiency  rating  is 
63%  and  scale  efficiency  is  97%  for  FY  1988. 

The  average  technical  efficiency  rating  for  all 
stores  over  the  two-year  period  of  this  study  is  in 
the  mid  60  percent  range  and  the  average  scale 
efficiency  rating  is  in  the  90  percent  range  except 
for  the  Navy  (see  Table  2).  Thus,  based  on  the 
variables  used  in  this  study,  improvements  can 
be  made  in  the  technical  and  scale  efficiencies 
of  all  the  commissaries  with  much  less 
improvement  evident  in  the  scale  of  operations. 

However,  we  do  not  want  to  overemphasize 
what  appears  to  be  a  low  average  technical  effi¬ 
ciency  rating.  Recall  that  it  is  a  relative  rating 
based  on  the  best  performing  commissaries. 
Senior  commissary  management  may  deem 
these  superior  performers  as  not  being  represen¬ 
tative  of  normal  or  expected  operations  and 
therefore,  may  set  lower  performance  goals  than 
indicated  by  DEA. 


store  Size 

The  relationship  between  store  size  and  the 
efficiency  ratings  is  examined  to  determine  if 
certain  sizes  of  stores  are  better  performers  or 
more  efficient  than  others.  Do  larger  commis¬ 
saries  perform  better  than  smaller  commissaries? 

For  this  review,  two  input  measures  of  size 
are  used — store  square  footage  and  amount  of 
appropriated  fund  (APF)  usage.  Sales  are  not 
used  as  a  measure  of  size  since  they  are  an  out¬ 
put  variable  in  our  model.  The  first  test  used  is 
univariate  correlation  analysis.  In  this  test,  the 
DEA  efficiency  ratings  are  regressed  against 
square  footage  and  APF  for  each  commissary  for 
each  year  to  determine  whether  there  is  a  rela¬ 
tionship  between  size  of  the  operations  and  effi¬ 
ciency  of  operations,  i.e,,  as  stores  become  larger 
(smaller)  do  they  become  more  or  less  efficient. 
Table  3,  which  provides  the  values  for  these 
tests,  shows  that  the  correlation  analysis  did  not 
reveal  a  strong  relationship  between  efficiency 
and  either  APF  or  store  square  footage  for  any  of 
the  Services,  Thus,  a  linear  relationship  between 
the  DEA  efficiency  ratings  and  the  size  of 
operations  is  not  apparent. 


TABLE  3 

R2  VALUES  for  CORRELATION  ANALYSIS 


Air  Force  Army  Navy  Marines 
Technical  Efficiency 


APF  .32 

Square  Footage  .24 

Scale  Efficiency 

APF  .22 

Square  Footage  .31 


.17  .21  .25 

.11  .07  .36 


.34  .36  .54 

.43  .45  .67 


Since  the  univariate  correlation  analysis  fails 
to  show  any  relationship,  we  check  for  square 
footage  and  APF  thresholds  at  which  efficiency 
is  significantly  different.  A  statistical  approach 
provided  by  Banker  [1989  and  1993]  is  used  to 
test  whether  there  is  a  difference  in  the  efficiency 
ratings  between  two  groups  of  commissaries 
under  consideration,  i.e.,  larger  commissaries 
versus  smaller  commissaries.  The  test  statistic,  as 
indicated  below,  is  a  measure  of  the  variance 
around  the  reciprocal  of  the  inefficiency  ratings. 
Banker  also  indicates  that  the  DEA  inefficiency 
rating  follows  either  a  half-normal  or  exponen¬ 
tial  distribution.  We  assume  that  the  perfor¬ 
mance  ratings  follow  an  exponential  distribution 
because  it  has  the  appropriate  skewness  and 
nonnegativity  characteristics  associated  with  the 
DEA  scalar  measure. 

The  test  statistic  in  this  case  is  given  by: 

(3)  [  y  {MV:  -  i)/ni]/[  y  (lA^.  - 1)/«2] 

where: 


=  The  efficiency  rating  computed  by  the  DEA 
model  for  each  commissary  in  the  reference 
group 

=  The  number  of  commissaries  included  in 
group  1  (Nj) 

^2  The  number  of  commissaries  included  in 
group  2  (N2) 
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This  test  statistic  follows  an  F-distribution 
with  (2np  2772)  degrees  of  freedom.  The  justifica- 
tion  for  this  test  statistic  can  be  found  in  Banker 
[19931. 

No  APF  threshold  could  be  identified  at 
which  efficiency  ratings  are  significantly  differ¬ 
ent.  However,  the  findings  for  square  footage  are 
different.  The  greatest  difference  in  performance 
ratings  exist  at  the  20,000  square  foot  threshold. 
The  stores  with  greater  than  20,000  square  feet 
are  much  more  efficient  at  a  level  of  significance 
of  less  than  .00001  for  both  technical  and  scale 
efficiency.  We  further  subdivided  the  upper  level 
into  a  20,000  to  30,000  square  feet  segment  and 
another  segment  for  greater  than  30,000  square 
feet.  We  found  that  the  stores  with  20,000  to 
30,000  square  feet  are  more  technically  and  scale 
efficient  than  those  stores  with  under  20,000 
square  feet  (p  =  .0021  and  p  <  .00001  respec¬ 
tively).  Those  stores  with  greater  than  30,000 
square  feet  are  more  technically  and  scale  effi¬ 
cient  than  stores  with  20,000  to  30,000  square  feet 
(p  =  .0543  and  p  <  .00001  respectively).  Thus,  the 
evidence  suggests  that  large  stores  are  generally 
both  more  technically  and  scale  efficient  than 
small  stores.  However,  we  would  not  want  to 
conclude  that  all  stores  should  be  greater  than 
30,000  square  feet.  There  could  be  other  factors 
not  captured  with  our  limited  number  of  vari¬ 
ables  which  would  influence  that  decision.  For 
example,  stores  of  that  size  would  probably  be 
too  large  for  the  customer  base  at  smaller  instal¬ 
lations.  A  measure  of  market  size  is  discussed 
later  in  the  paper. 

These  results  also  suggest  that  standardized 
policies  and  procedures  for  all  stores  across  the 
Department  of  Defense  may  not  be  appropriate. 
Some  smaller  commissaries,  e.g..  Fort  Story  and 
Hill  AFB,  were  considered  efficient.  The  policies 
and  procedures  of  these  efficient  smaller  com¬ 
missaries  could  be  reviewed  for  possible 
adoption  for  smaller  commissaries. 

Sensitivity  Analysis 

Using  Air  Force  data  only  (because  of  avail¬ 
ability),  the  sensitivity  of  the  individual  commis¬ 
sary  performance  results  to  the  addition  of 
different  inputs  and  outputs  is  examined.  One 
variable  tested,  both  as  an  output  and  an  input, 
is  average  weekly  operating  hours.  Operating 


hours,  as  an  output  variable,  serves  as  a  surro¬ 
gate  for  customer  service.  We  would  expect  that 
the  greater  the  operating  hours  the  greater  the 
service  to  the  customer  and  the  greater  the 
inputs,  i.e.,  work-years  and  appropriated  funds, 
consumed.  Another  analysis  is  accomplished 
using  operating  hours  as  an  input — the  more 
hours  the  store  is  open  the  greater  the  sales 
should  be.  Surprisingly,  neither  case  made  much 
difference  in  the  efficiency  ratings.  Possibly,  the 
information  provided  by  operating  hours  is 
being  captured  in  the  other  output  and  inputs. 

A  third  sensitivity  analysis  is  done  using  a 
measure  of  market  size — the  number  of  active 
duty  military  and  retirees  in  the  area — as  an 
input.  The  population  numbers  were  obtained 
from  Economic  Resource  Impact  Statements 
(ERIS)  prepared  by  each  Air  Force  base.  This 
analysis  did  change  the  efficiency  ratings  of  Air 
Force  stores — with  some  stores  getting  better 
ratings  while  the  ratings  for  other  stores  deterio¬ 
rated.  However,  there  was  no  pattern  to  these 
changes.  This  indicates  that  the  DEA  perfor¬ 
mance  ratings  are  sensitive  to  and  influenced  by 
market  size.  Unfortunately,  this  information  is 
not  readily  available  for  Army,  Navy,  and 
Marine  Corps  installations  so  no  further  analysis 
is  pursued. 

Future  Research  Relative  to 
Commissaries 

There  are  two  areas  of  study  that  the 
Defense  Commissary  Agency  (DECA)  may  wish 
to  pursue  based  on  this  research.  First,  a  longitu¬ 
dinal  analysis  would  facilitate  evaluating 
whether  efficiencies  and  economies  were 
achieved  from  the  consolidation  of  the  four  mili¬ 
tary  commissary  systems  into  one  DoD  system. 
A  second  potential  area  of  study  is  in  developing 
additional  output  and  input  measures.  In  this 
research,  output  and  input  variables  on  which 
data  were  currently  being  gathered,  reported, 
and  accepted  as  useful  by  management  were 
used.  However,  some  key  success  factors  may 
have  been  missed.  For  example,  additional  out¬ 
puts  better  reflecting  customer  service  could  be 
developed  and  tracked.  On  the  input  side,  a 
measure  of  market  size  might  be  useful,  as 
indicated  earlier  in  this  study. 
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VI.  CONCLUSIONS 

This  study  evaluated  a  benchmarking 
approach  for  a  DoD  business-like,  service 
activity.  It  served  two  purposes.  First,  the  bench¬ 
marks  identified  provide  possible  avenues  for 
improving  commissary  operations.  Since  com¬ 
missary  operations  are  a  far  more  complex 
process  than  represented  by  this  four  variable 
model,  the  benchmarks  indicate  the  direction 
that  needs  to  be  pursued  rather  than  specific 
operational  metrics  that  are  immediately  attain¬ 
able.  Also,  the  data  used  are  from  1988-89  and 
may  not  be  representative  of  today's  operations. 
Department  of  Defense  business-like  activities 
such  as  commissaries  need  to  be  operated  in  an 
efficient  manner  so  as  to  insure  economical  use 
of  taxpayers'  dollars,  preserve  the  military  mem¬ 
bers'  benefits  these  dollars  provide,  and  mini¬ 
mize  military  member  and  dependent  costs 
(prices,  fees,  dues,  etc.). 

Second,  the  research  illustrates  and  evalu¬ 
ates  the  potential  for  using  data  envelopment 
analysis  (DBA)  as  a  benchmarking  or  "best-prac¬ 
tices"  tool  in  order  to  improve  performance  and 
provide  better  management  and  cost  control. 
DBA  is  an  objective  means  for  selecting  bench¬ 
mark  partners,  quantifying  performance  gaps, 
and  identifying  information  which  might  be 
useful  for  improving  performance. 

DEA  as  a  Benchmark  Identifier 

Benchmarking  or  best-practices  analysis  is 
an  integral  aspect  of  total  quality  management.  It 
requires  the  identification  and  observation  of 
best  practices  and  then  projecting  future  perfor¬ 
mance  based  on  adopting  these  best  practices. 
Thus,  this  process  involves,  among  other  actions, 
identifying  benchmark  partners  (i.e.,  organiza¬ 
tions  with  superior  performance)  as  a  source  of 
comparative  data  or  benchmarks,  gathering  per¬ 
formance  data,  determining  any  performance 
gap,  projecting  superior  performance  levels,  and 
establishing  realistic,  achievable  performance 
goals  [Camp,  1989].  Are  the  benchmark  partners 
better?  By  how  much?  What  level  of 
performance  should  we  realistically  expect? 

In  the  case  presented  here,  superior  perfor¬ 
mance  is  identified  via  DBA.  Then  through 
using  DBA,  performance  gaps  are  quantified 


and  potential  benchmarks  for  a  commissary's 
continued  improvement  are  provided.  DBA  indi¬ 
cates  where  improvements  are  necessary;  e.g., 
less  work-years,  reduced  appropriated  fund 
expenditures,  and  less  store  square  footage;  and 
the  magnitude  of  these  improvements.  In  addi¬ 
tion,  DBA  provides  the  benchmark  units  (stores 
operating  at  100%  efficiency)  upon  which  a  less 
efficient  store's  performance  is  determined. 
These  comparison  stores  have  a  similar  pro¬ 
duction  function  and  can  serve  as  benchmark 
partners  for  identifying  best  practices  from 
which  the  inefficient  store  can  get  ideas  for 
improvement. 

In  addition,  by  benchmarking  the  "best-of- 
class,"  the  Defense  Commissary  Agency  can 
focus  on  continuous  improvement  of  processes 
and  activities  that  support  these  processes.  For 
example,  this  research  indicates  that  larger  stores 
(greater  than  20,000  square  feet)  are  significantly 
more  efficient  than  smaller  stores.  Hence,  DBCA 
may  wish  to  initially  focus  its  efforts  on  the  small 
commissaries  in  order  to  improve  system  effi¬ 
ciency.  There  are  some  small  commissaries  that 
are  efficient.  The  operating  procedures  of  these 
commissaries  should  be  examined  and  possibly 
adopted  for  use  by  the  other  small  less  efficient 
stores — even  if  they  aren't  in  the  comparison  set 
of  the  inefficient  store. 

Finally,  a  key  aspect  of  the  benchmarking 
process  is  gaining  management  acceptance  of 
the  findings.  Contributing  to  this  acceptance  is 
benchmarks  that  are  relevant  and  current.  DBA 
provides  objective,  empirically-based  per¬ 
formance  evaluations  based  on  attained  perfor¬ 
mance.  Thus,  through  DBA,  benchmark  infor¬ 
mation  can  be  collected  that  will  allow  setting 
performance  objectives  that  are  realistic  and 
attainable.  Furthermore,  studies  have  shown  via 
theoretically  constructed  (artificially  generated) 
data  and  actual  operational  data  that  DBA  is  able 
to  identify  superior  performance  and  establish 
realistic  performance  goals  [e.g..  Banker  et.  d., 
1988  and  Bowlin  et.  ^.,  1985]. 
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ABSTRACT 

Computer  programs  that  simulate  war¬ 
fare  generally  do  not  model  bomb  damage 
assessment  (BDA)  in  a  complex  fashion.  Most 
computer  models  either  assume  perfect  BDA 
or  no  BDA,  or  allow  the  user  to  select  be¬ 
tween  these  two  options  (sometimes  globally 
and  sometimes  on  a  target-by-target  basis). 
Assuming  perfect  BDA  will  allow  a  given 
force  to  destroy  a  given  target  set  much  more 
rapidly  than  will  be  the  case  if  there  is  no 
BDA,  because  perfect  BDA  prevents  the 
waste  of  weapons  on  targets  that  have 
already  been  destroyed.  It  would  super¬ 
ficially  appear  that  perfect  BDA  and  no  BDA 
are  opposite  limiting  extremes.  This  is  not  the 
case.  The  opposite  limiting  extremes  are  actu¬ 
ally  perfect  BDA  and  extremely  bad  BDA. 
''No  BDA"  is  an  intermediate  case,  although 
probably  closer  to  "worst  case"  than  to  "best 
case"  in  most  realistic  battlefield  scenarios. 
Hence,  it  is  probably  more  reasonable  to 
assume  no  BDA  than  perfect  BDA  in  doing 
computer  modeling.  It  is  easy  to  simulate  per¬ 
fect  BDA  or  nonexistent  BDA,  but  it  is  very 
hard  to  model  flawed  BDA.  Any  method  of 
modeling  imperfect  BDA  will  be  highly 
dependent  on  the  scenario  assumed. 

INTRODUCTION 

Computer  programs  that  simulate  war¬ 
fare  generally  do  not  model  BDA  in  a  com¬ 
plex  fashion.  Most  computer  models  either 
assume  perfect  BDA  or  no  BDA,  or  allow  the 
user  to  select  between  these  two  options 
(sometimes  globally  and  sometimes  on  a  tar- 
get-by-target  basis).  Assuming  perfect  BDA 
will  allow  a  given  force  to  destroy  a  given 
target  set  much  more  rapidly  than  will  be 
the  case  if  there  is  no  BDA,  because  perfect 
BDA  prevents  the  waste  of  weapons  on  tar¬ 
gets  that  have  already  been  destroyed.  In 
this  paper,  we  attempt  to  develop  a  mathe¬ 
matical  formalism  for  determining  the 
effects  that  BDA  can  have  on  the  number  of 
sorties  launched  against  a  given  target  set. 
We  also  address  the  following  two 
questions: 

•  Are  perfect  BDA  and  no  BDA  oppo¬ 
site  limiting  extremes? 

•  Is  it  generally  more  reasonable  to 
assume  perfect  BDA  or  no  BDA  in  a 
computer  simulation  of  war? 


In  order  to  facilitate  this  analysis,  we 
examine  two  variations  on  one  basic 
scenario,  which  is  described  below. 

BASIC  SCENARIO: 

Suppose  that  we  have  live  identical 
point  targets  initially  present.  We  launch  n 
raids  against  these  targets,  where  n  is  con¬ 
strained  to  be  a  positive  integer.  In  each  raid, 
there  is  one  sortie  against  each  target  that  is 
thought  to  be  "live."  All  sorties  are  identical. 
On  the  first  raid,  all  targets  are  accurately 
known  to  be  alive,  so  there  is  one  sortie 
against  each  of  the  targets,  for  a  total  of 

sorties.  Let  P  denote  the  probability  that 
any  one  sortie  kills  the  target  that  it  is  attack¬ 
ing.  We  constrain  P  to  be  less  than  one;  oth¬ 
erwise  the  problem  is  trivial  with  all 
targets  being  destroyed  on  the  first  raid. 
Beginning  after  the  first  raid,  there  is  a 
reconnaissance  flight  to  assess  the  damage 
done  on  the  previous  raid.  The  first  recon¬ 
naissance  flight  will  examine  all  targets, 
because  all  N^  targets  were  attacked  on  the 
first  raid. 


No- 

ki- 

P  = 

P  = 
1-p- 

q  = 

l-q- 


Make  the  following  definitions: 
number  of  targets  initially  present 
expected  number  of  targets  still 
"living"  after  j  raids 
expected  number  of  targets  killed  on 
raid  number  i 

expected  cumulative  number  of 
targets  killed  in  first  j  raids  =  -  N- 

single-sortie  kill  probability  (one 
aircraft  versus  one  target) 
probability  of  wrongly  assessing  a 
dead  target  as  being  living 
probability  of  correctly  assessing  a 
dead  target  as  being  dead 
probability  of  wrongly  assessing  a 
live  target  as  being  dead 
probability  of  correctly  assessing  a 
live  target  as  being  live 
expected  target  set  remaining  after 
raid  j  =  expected  target  set  for  raid 
j+1  =  expected  number  of  sorties  on 
raid  j+l 

expected  number  of  live  targets 
assessed  as  alive  after  j  raids 
expected  number  of  dead  targets 
assessed  as  alive  after  j  raids 
expected  total  number  of  sorties 
flown  in  first  j  raids 
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It  is  immediately  clear  that: 


To  =  A„ 


:N 


O 


K-  =  ^  fc,-  =  = 

^  i=l  1=1 

N^Pll-il-qKl-Py]  (8) 

1-(1-<7)(1-P) 

Letting  the  number  of  raids  approach  infinity, 
we  find  that: 


ko  =  0 

Kj  =  k;  =  N„P 

Ni  =  N„(l-P)  =  N„-kj 


lim  ^  - NqP - 

j^oo  ’  1 -(1 -<?)(! -P) 

The  value  of  Kj  can  reach  only  if  q  =  0.  If  q  -  1, 
only  the  first  raid  destroys  any  targets. 


is  the  target  set  for  raid  number  two.  con¬ 
sists  of  that  portion  of  which  is  correctly 
assessed  as  being  alive  plus  that  portion  of 
which  is  incorrectly  assessed  as  being  alive.  In 
other  words,  +  D^.  We  find  that: 


We  also  have  the  following  expression  for 
D.  =  ^  =  NgPy  (1  -P)y-'  (10) 

^  i=1  /=0 


Ti  =  No(l-P)(l-^7)  PN/p  (3) 

Things  get  more  complicated  when  we  come  to 
the  second  and  later  raids.  We  find  that: 

T.  =  A.  +  D.  (4) 

If  p  =  0,  then  Dj  =  0  and  Tj  =  Aj. 


p'-(i-<7)/(i-py 


This  simplifies  to: 

D,.  =  NpPp , 

Using  Equations  (3),  (5),  and  (11),  we  find  that: 

pi-ii-qm-py 


(11) 


Tj  =  No{l-Pni-q)i  + 


p-(l-q)(l-P) 


(12) 


kj  =  PAj_.^  ■  (5) 

The  remainder  of  the  analysis  depends  on 
what  happens  after  each  reconnaissance  flight. 

EXAMPLE  ONE 


Rearranging  Equation  (12)  and  using  Equation 
(2),  we  get: 

r  cv  -i 


NpPp 

p-il-qKl-P) 


X  (1 -<?)'(! -P)' + 

/=0 


Two  basic  limiting  cases  are  possible.  In  the 
limiting  case  considered  here,  any  target  that  is  (right¬ 
ly  or  wrongly)  assessed  as  "dead"  after  any  raid  will 
never  he  assessed  or  attacked  again.  Any  target  that 
is  assessed  as  "live"  after  all  raids  to  date  will  be 
attacked  again,  by  one  sortie,  on  the  next  raid. 
Generally  speaking,  the  number  of  sorties  per 
raid  will  decline  as  the  number  of  targets 
assessed  to  be  "live"  decreases.^  In  this  limiting 
case,  we  find  that: 


NpPp 

p-(l-q)0-"P) 


i=0 


(13) 


Evaluating  the  two  summations,  we  get  the  fol¬ 
lowing  expression  for  S-: 


N _ m. . . 

0  p-(l-^?)(l-P) 


l-(l-(?y(l-P); 

l-(l-^)l-p) 


Aj  =  Noa-qya-py  (6) 

kj  =  PAj_.i  =  NgPa -qy-^a -py-^  (?) 


NpPp 

p-il-qXl-P) 


l-p 

l-p 


(14) 
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In  order  to  determine  the  expected  number 
of  sorties  required  to  achieve  some  damage  goal, 
G,  the  first  step  is  to  set  =  G  in  Equation  (8) 
and  then  solve  for  n.  (If  G  is  greater  than  the  lim¬ 
iting  value  of  from  Equation  (9),  then  it  is 
impossible  to  achieve  the  goal  and  this  proce¬ 
dure  will  not  work.)  This  determines  the 
expected  number,  n,  of  raids  required  to  reach 
the  goal  The  result  is: 


In 


n  = 


NoP-Ga-(l-^)(l-P)) 


(15) 


kj  = 

K:  =  NoP^(l-P)'-’  =  Nod  -  (l-py)  =  PS.  (17) 

As  the  number  of  raids  goes  to  infinity,  Kj  goes  to 
Nq.  Moreover,  no  weapons  are  ever  wasted  on 
dead  targets  and  the  ratio  between  the  number 
of  targets  destroyed  and  the  number  of  sorties 
flown  is  simply  R 

Limiting  case:  q  =  0  and  p  =  1  (no  BDA) 

If  q  =  0,  there  is  no  major  simplification 
unless  p  =  1.  If  p  -  1,  there  are  sorties  in  every 
raid  and: 


The  value  of  n  predicted  by  Equation  (15)  will 
generally  not  be  an  integer;  it  should  be  rounded 
up.  The  next  step  is  to  use  Equation  (14)  to 
determine  the  corresponding  number  of  sorties. 


S-/No  (18) 

K-  =  NoP^(l-P)'-i  =  Ngd  -  (l-py)  (19) 
^  /=! 


Finally,  we  use  Equations  (8)  and  (14)  to 
develop  a  relationship  between  Kj  and  S-.  The 
resulting  expression  is: 


pp-m-ci)(i-p) 

Q  - 

m  1 

p-pp-n-cjXi-p) 

p-a-ci)a-p) ' 

1. 

7/p  =  0,  then  Equation  (16)  reduces  to  Kj  =  PSj.  If  q  = 
0  but  p  >  0,  this  does  not  lead  to  any  major 
simplification. 

NOTE:  In  the  preceding  analysis,  Tj,  K^,  and  S- 
can  assume  nonintegral  values.  In  practice,  when 
doing  computer  simulations  of  war,  it  might  be 
reasonable  to  round  Tj  up  or  down  to  the  nearest 
integer  after  every  raid.  Similarly,  the  expression 
for  Sj  should  probably  be  rounded  off  to  the 
nearest  integer.  Finally,  it  would  probably  be  rea¬ 
sonable  to  classify  the  entire  target  set  as  dead 
once  N-  drops  below  1.  Of  course,  it  is  usually 
the  case  that  the  goal  is  to  kill  something  like 
50%  or  80%  of  the  target  set,  so  the  question  of 
how  a  computer  program  handles  the  last  target, 
out  of  several  thousand,  is  probably  not 
important. 

Best  possible  case:  p  =  q  =  0  (perfect  BDA) 

In  this  case,  we  find  that: 

r.  =  N^(i-py 


The  expected  fractional  number  of  raids,  n, 
needed  to  destroy  a  goal  of  G  targets  is  given  by: 


n  = 


In 


\ 


_G 

No/ 


In(l-P) 


(20) 


The  value  of  n  from  Equation  (20)  should  be 
rounded  up.  The  corresponding  number  of  sor¬ 
ties  is  just  uNq.  As  the  number  of  raids  ap¬ 
proaches  infinity,  K-  goes  to  and  all  targets  are 
destroyed.  However,  some  sorties  are  wasted  on 
dead  targets. 

Worst  case:  q  =  1  and  p  =  1 

In  this  case,  N^P  targets  are  destroyed  in  the 
first  raid.  After  the  first  raid,  all  dead  targets  are 
erroneously  classified  as  being  alive  and  all  live 
targets  are  erroneously  classified  as  being  dead. 
All  targets  that  survived  the  first  raid  are  im¬ 
mune  from  further  attacks,  so  the  number  of  tar¬ 
gets  killed  remains  stuck  at  N^P  no  matter  how 
many  raids  are  launched. 

EXAMPLE  TWO 

We  now  consider  a  different  limiting  case. 
Beginning  after  the  first  raid,  there  is  a  reconnais¬ 
sance  flight  to  assess  the  status  of  all  targets.  Any 
target  that  is  assessed  as  "dead"  after  any  raid  will 
not  be  attacked  on  the  next  raid.  Any  target  that  is 
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assessed  as  "live"  after  any  raid  will  be  attacked 
again,  by  one  sortie,  on  the  next  raid.  In  this  limit, 
a  target  that  is  assessed  as  being  dead  can  get  hack 
into  the  target  set  at  a  later  time.  This  means  that 
the  target  set  will  not  automatically  shrink  from 
raid  to  raid.  Equations  (1)  through  (5)  from  the 
Basic  Scenario  are  still  valid.  However,  we  have 
the  following  relationships,  which  were  different 
from  those  used  in  Example  One: 


D.^^pK. 

(21) 

A.  =  (l-q)N. 

(22) 

i 

1 

11 

(23) 

Equations  (22)  and  (23)  immediately  imply  that: 


T.  =  %  +  Nod -P)[l -P(l -q)rHl -q-p),  for  j  >  1 

(31) 

Using  Equations  (2)  and  (31),  plus  the  expression 
for  Tq,  we  find  that: 


1  +  pij-l)  +  (\-P){\-q-p)\ 

r 

i-d-p(i-(?)y-i 

1  P{l-q)  l\ 

(32) 

In  order  to  determine  the  expected  fractional 
number  of  sorties  required  to  achieve  some  dam¬ 
age  goal,  G,  the  first  step  is  to  set  =  G  in 
Equation  (30)  and  then  solve  for  n.  This  deter¬ 
mines  the  expected  number,  n,  of  raids  required 
to  reach  the  goal.  The  result  is: 


Nj  =  (1  -  P(l-c]))Nj^^,forj  >  1  (24) 

We  know  that  =  (1  -  P)Np,  so  Equation  (24) 
immediately  leads  to: 

N.  -  Nod  -  P)[l  -  Pil-q)r\for  j  >  1  (25) 

A.  =  Nod  -  q)(l  -  P)[l  -  P{l-q)]r\forj  ^  1 

(26) 

Kj  =  hi  =  +  hi  =  PNo  +  Phi~i  (27) 

The  previous  equation  leads  to: 

7-1  7-2 

K:=PNo+P^Ai=PN^+N^P{\  -P)(l  -(?)y  (1  -P(l  -q)y 

,=i  .=0  (28) 


K,.=PNo+PNo(l-P)(l-<?) 


i-(i-Pii-q)y-^ 


pa-q) 


(29) 


The  previous  equation  simplifies  to: 

Kj  =Nq-  Nod  -P)(l -P(l -q))’-^  =  No  -  N-  (30) 


n  =  1  + 


In 


Nq-G 

Npd-P) 


\n{l-P{l-q)) 


(33) 


The  value  of  n  from  Equation  (33)  should  be 
rounded  up.  The  next  step  is  to  use  Equation  (32) 
to  determine  the  corresponding  number  of 
sorties. 

Finally,  we  use  Equations  (30)  and  (32)  to 
develop  a  relationship  between  Kj  and  Sj.  The 
result  is: 


K^.  =  PNo  + 


Pd-?) 


(s.  +  No(p-yp-i)) 


(34) 


If  p  =  0  and  q  >  0,  then  IC  =  PSj  for  all  q,  but 
Equation  (30)  for  Kj  and  E^quation  (33)  for  the 
number  of  raids  required  do  not  simplify  any, 
and  Equation  (32)  for  Sj  does  not  simplify  much. 
If  q  =  0,  then  there  is  no  major  simplification 
except  when  p  =  1. 


In  this  scenario,  the  number  of  targets  killed 
approaches  as  the  number  of  raids  approaches 
infinity,  provided  that  q  <1,  because  a  live  target  that 
is  erroneously  classed  as  dead  will  eventually  get  back 
into  the  target  set.  If  we  use  the  results  of 
Equations  (30)  and  (25),  we  find  that  Kj  -i-  Nj  = 
Nq,  which  is  a  requirement  that  must  be  satis¬ 
fied.  Equations  (4),  (21),  (26),  and  (30)  imply  that: 


NOTE:  In  the  preceding  analysis,  Tj,  K.,  and  Sj 
can  assume  nonintegral  values.  Hence,  tne  same 
caveats  that  were  expressed  about  fractional 
sorties  in  Example  One  also  apply  here. 

Best  possible  case:  p  =  q  =  0  (perfect  BDA) 

In  this  case,  the  limiting  behavior  is  identical 
to  that  in  Example  One.  Because  no  weapons  are 
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ever  wasted  on  dead  targets  whenever  p  =  0,  the 
ratio  between  the  number  of  sorties  flown  and 
the  number  of  targets  destroyed  is  simply  P. 

Limiting  case:  q  =  0,  p  =  1  (no  BDA) 

In  this  case,  Equations  (18)  through  (20)  from 
Example  One  are  still  valid.  If  q  =  0  and  0  <  p 
<  1,  there  is  no  major  simplification  and  the  lim¬ 
iting  behavior  is  different  from  that  in  Example 
One. 


Worst  case:  q  =  1  and  p  =  1 

In  this  case,  N^P  targets  are  destroyed  in  the 
first  raid  and  no  targets  are  destroyed  in  the  later 
raids,  no  matter  how  many  raids  are  launched. 

CONCLUSIONS 

1)  The  opposite  limiting  extremes  are  not  "per¬ 
fect  BDA"  and  "no  BDA"  The  opposite  extremes  are 
perfect  BDA  and  extremely  bad  BDA,  "No  BDA"  is 
an  intermediate  case,  although  probably  closer  to 
"worst  case"  than  to  "best  case"  in  most  realistic 
battlefield  scenarios. 

2)  Most  computer  models  assume  either  perfect 
BDA  or  no  BDA,  or  allow  the  user  to  choose  between 
these  two  options.  Given  that  "no  BDA"  is  an  inter¬ 
mediate  case,  whereas  "perfect  BDA"  is  an  extreme 
limit,  it  is  generally  more  reasonable  to  choose  "no 
BDA"  if  that  option  is  available. 

3)  It  is  easy  to  simulate  perfect  BDA  or  nonex¬ 
istent  BDA,  but  it  is  hard  to  model  flawed  BDA.  Any 
method  of  modeling  imperfect  BDA  will  be  highly 
dependent  on  the  scenario  assumed. 


4)  Of  the  computer  models  known  to  the 
author,  only  HEAVY  ATTACK  allows  the  user  to 
select  intermediate  BDA.  HEAVY  ATTACK  con¬ 
tains  a  BDA  factor,  y,  that  can  be  varied  from 
zero  to  one,  on  a  global  basis.  The  HEAVY 
ATTACK  damage  accumulation  formula  is: 

«pi/\ 


L.  \ 


H 


\y  I 


l-e 


(35) 


\ 


In  Equation  (35),  n  is  the  number  of  sorties. 
In  HEAVY  ATTACK,  each  aircraft  is  limited  to 
attacking  one  target,  so  the  expected  damage 
from  each  sortie  is  just  P.  If  individual  aircraft 
could  attack  more  than  one  target,  then  P  would 


be  replaced  by  the  total  expected  damage  proba¬ 
bility  per  sortie.  If  y  =  0,  then  Equation  (35)  leads 
to  linear  damage  addition  (K^^  =  nP),  which  is 
what  we  would  expect  with  perfect  BDA.  If  y  >  0 
in  HEAVY  ATTACK,  then  damage  accumulates 
more  slowly  than  is  the  case  if  y  =  0,  but  there  is 
no  obvious  relationship  between  y  and  any  of 
the  parameters  in  this  analysis.  In  fact,  y  may 
have  no  physical  meaning;  it  may  simply  be  an 
arbitrary  factor  that  emulates  the  effects  of  poor 
BDA  by  making  damage  accumulate  more  slow¬ 
ly  than  would  otherwise  be  the  case.  For  exam¬ 
ple,  a  nonzero  value  of  y  reduces  the  amount  of 
damage  done  on  all  raids,  including  the  first 
raid.  In  reality,  poor  BDA  should  reduce  the 
efficiency  of  later  raids,  but  not  the  first  raid.^ 

ENDNOTES 

^  An  alternative  approach  would  be  to  keep  the 
number  of  sorties  per  raid  constant.  This 
means  that  each  target  that  is  assessed  as  being 
"live"  after  raid  j  would  tend  to  be  hit  harder 
on  raid  j+1  than  on  the  previous  raid.  We  do 
not  consider  this  alternative,  because  it  would 
quickly  lead  to  fractional  sorties  against  each 
target  assessed  as  live. 

^  Survey  and  Description  of  USAF  Conventional 
Munitions  Allocation  Models,  Major  Kirk  A. 
Yost,  Office  of  Aerospace  Studies,  Directorate 
of  Requirements,  Air  Force  Materiel 
Command,  5  December  1994 
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The  purpose  of  MORS  is  “to  enhance  the  quality  and 
effectiveness  of  classified  and  unclassified  military  operations 
research.”  A  key  medium  for  accomplishing  that  purpose  is 
professional  publications  such  as  Military  Modeling  for 
Decisions.  This  monograph  is  the  latest  step  in  MORS’ 
dedication  to  furthering  the  technical  base  and  professionalism 
of  our  membership  and  the  military  operations  research 
community  at  large. 

From  its  beginning,  military  operations  research  has  aimed  to 
help  decision  makers  choose  more  optimal  courses  of  action. 
Models  have  been  the  mainstay  of  our  toolkit,  analysis  is  our 
methodology,  and  better  decisions  are  our  objective.  In  1984, 
Professor  Wayne  Hughes  edited  the  first  edition  of  Military 
Modeling.  The  purpose  of  this  monograph  was  to  enhance  the 
analytical  tools  knowledge  of  our  current  members  and  to 
provide  a  collated  reference  for  the  young  analysts  yet  to  join 
our  ranks.  It  was  a  compilation  of  chapters  written  by  leaders  of 
the  military  operations  research  community  discussing  the 
development  and  application  of  models  for  military  decision 
making.  By  1989,  MORS  had  developed  a  taxonomy  of  models 
and  simulations,  verification  and  validation  had  been  clarified 
by  a  series  of  MORS  workshops,  and  the  first  edition  had 
almost  sold  out.  Therefore,  Professor  Hughes  led  the 
preparation  of  a  second  edition. 

The  intervening  years  between  1989  and  today  have  been 
extremely  dynamic  for  the  US  military  and  its  MOR 
community.  Relative  to  military  modeling,  the  following  five 
attributes  were  significant: 


1.  Technologies  available  to  support  modeling  and 
analysis,  to  include  computing  power,  are  rapidly  expanding. 

2.  Military  training  and  education  communities  have 
embraced  the  use  of  modeling. 

3.  Operational  pace  and  breadth  has  greatly  increased  with 
a  resultant  increase  in  operational  and  combat  applications  of 
modeling. 

4.  Development  of  a  theory  of  combat  has  progressed. 

5.  Military  organizations  to  develop  and  apply  models 
have  been  created  and/or  expanded. 

As  a  result  of  these  factors  and  a  dwindling  supply  of  the 
second  edition,  MORS  once  more  called  Professor  Hughes  to 
duty  as  editor.  This  third  edition  has  been  retitled  as  a  result  of 
concerns  that  the  tools  can  sometimes  capture  the  imagination 
and  attention  of  the  analyst  to  the  detriment  of  the  analysis  itself 
(see  the  quotes  below).  As  a  result,  the  authors  and  editors  of 
the  third  edition  are  paying  particular  attention  to  the 
apphcations  of  military  modeling  for  decisions  in  the  main 
domains  of  military  endeavor.  This  edition  has  significantly 
increased  the  coverage  of  operational  uses  of  models  and  has 
added  chapters  on  training  analysis  and  VV&A.  The  product  is 
noteworthy  in  capturing  the  uses  of  modeling  in  recent 
operations  such  as  DESERT  STORM  and  in  relating  the 
potential  and  pitfalls  of  some  of  the  new  modeling  technologies. 


Selected  Quotes  from  Military  Modeling  for  Decisions 

•  Since  the  model  tail  sometimes  seems  to  wag  the  analysis  dog,  there  is  a  need  to  sharpen  the  focus  on  the  core  of 
our  profession  ...  making  better  decisions. 
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ABSTRACT 

his  paper  presents  an  application  of 
bivariate  probability  theory  to  model¬ 
ing  cost  and  schedule  uncertainties.  It 
has  long  been  recognized  that  program  cost 
and  schedule  are  correlated;  however,  for¬ 
mal  methods  have  not  been  developed  in 
the  cost  analysis  community  to  study  their 
joint  behavior.  To  address  this,  bivariate 
models  for  approximating  the  joint  and  con¬ 
ditional  probabilities  of  program  cost  and 
schedule  are  presented.  Specifically,  the 
bivariate  lognormal  and  bivariate  normal- 
lognormal  models  are  discussed.  The 
statistical  properties  of  these  models  are  pro¬ 
vided.  A  cost  analysis  application  is  pre¬ 
sented  to  illustrate  their  use  in  a  practical 
context. 

I.  INTRODUCTION 

A  common  framework  for  quantifying 
the  cost  impacts  of  uncertainty  is  the  work 
breakdown  structure  (WBS)  [ll  The  WBS  is 
a  product-oriented  hierarchy  of  cost  ele¬ 


ments  that  define  the  program.  Illustrated  in 
table  1,  many  of  these  elements,  such  as 
system  engineering,  reflect  distinct  level-of- 
effort  activities  that  vary  over  a  program's 
schedule.  Thus,  the  uncertainty  associated 
with  total  program  cost  is  correlated  to  the 
uncertainty  in  the  overall  schedule. 

Numerous  methods  exist  for  quantifying 
cost  uncertainty  from  a  WBS  perspective. 
However,  they  are  focused  on  producing  a 
univariate  distribution  of  total  cost;  schedule 
uncertainties  are  not  explicit.  Cost-schedule 
questions  that  are  probability-based  (e.g., 
what  is  the  chance  of  delivering  the  system 
within  cost  and  schedule  estimates?)  are  not 
readily  answered  by  existing  models, 
methodologies,  or  spreadsheet  simulation 
applications. 

This  paper  extends  current  practice  by 
providing  bivariate  probability  models  from 
which  joint  and  conditional  distributions  of 
cost  and  schedule  can  be  developed.  This 
fosters  the  early  identification  of  cost-sched¬ 
ule  risks  such  that  mitigation  strategies  can 
be  planned  and  implemented  in  a  timely 
manner. 


Table  1.  An  Illustrative  WBS 

Cost  Element  (CE)  Estimation  Heuristic 

(Example  Only) 

Prime  Mission  Equipment  (PME)  Segment 


CEj 

Hardware  (HW) 

CEj 

Software  (SW) 

SWcgsi  =  8.2\, y^Source  Lines/1000)€ 

CE3 

Integration  &  Assembly  (I&A) 

^^^Cost 

^  Test  Equip 

System  Support  (SS)  Segment 

CE4 

System  Engineering  (SE) 

EEcosf  “  ^Pgrm^SE^^^hE 

CE, 

Program  Management  (PM) 

E^Cosi  ~  Epg^^.piy,(SL)p/yj 

CE, 

Test  &  Evaluation  (TE) 

^Ecosf  ~  ^Pgrm^TE^^^^TE 

CE. 

Training 

Trainingf-g^f  —  ^jrainin^^Cosl 

CE„ 

Data 

Notes:  denotes  a  labor  rate;  e  is  a  statistical  error  distribution;  ^l&cA  denotes  that  portion  of  the  overall  program  sched¬ 

ule  for  I&A  activities;  denotes  a  staff-level;  and  denotes  a  cost  factor. 
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II.  BACKGROUND  &  MODEL  DOMAIN 

This  section  provides  a  brief  background  on 
the  types  of  uncertainties  present  in  military  sys¬ 
tems  engineering  projects.  A  discussion  on  how 
these  uncertainties  relate  to  the  models 
described  in  this  paper  is  presented. 

A.  Background 

Uncertainty  analysis  had  its  genesis  in  a 
field  known  as  military  systems  analysis  [2], 
which  was  founded  in  the  1950's  at  the  RAND 
Corporation.  Shortly  after  World  War  11,  military 
systems  analysis  was  developed  to  aid  defense 
planners  with  long-range  decisions  on  force 
structure,  force  composition,  and  future  theaters 
of  operation.  Cost  became  a  critical  consideration 
in  military  systems  analysis  models  and  decision 
criteria.  However,  cost  estimates  of  future  mili¬ 
tary  systems,  particularly  in  the  early  acquisition 
phases,  were  often  significantly  lower  than  the 
actual  cost  or  an  estimate  developed  at  a  later 
phase.  In  Cost  Considerations  in  Systems  Analysis 
[3],  Gene  H.  Fisher  attributes  this  difference  to 
the  presence  of  uncertainty;  specifically,  cost  esti¬ 
mation  uncertainty  and  requirements  uncertain¬ 
ty- 

Cost  estimation  uncertainty  originates  from 
errors  in  cost-schedule  estimation  models,  from 
the  misuse  or  misinterpretation  of  cost-schedule 
data,  or  from  misapplied  cost-schedule  estima¬ 
tion  methodologies.  Economic  uncertainties  that 
affect  the  cost  of  technology  (e.g.,  infla¬ 
tion/deflation),  the  labor  force,  or  geo¬ 
political  policies  further  contributes  to  cost 
estimation  uncertainty. 

Requirements  uncertainty  originates  changes 
to  a  system's  specified  configuration  (i.e.,  the 
system  architecture).  Different  system  configura¬ 
tions  can  arise  from  changes  in  the  understood 
threat,  mission  objective,  or  strategic  outlook;  if 
the  system's  acquisition  approach  differs  sub¬ 
stantially  from  the  plan;  or  if  achieving  perfor¬ 
mance  requirements  necessitates  major 
alterations  in  the  system  configuration. 

Uncertainty  is  also  present  in  elements  that 
define  a  system's  configuration.  This  is  referred 
to  as  system  definition  uncertainty.  Examples 
include  uncertainties  in  the  amount  of  software 
to  develop,  the  extent  that  code  from  another 
system  can  be  reused,  the  number  of  worksta¬ 


tions  to  procure,  or  the  delivered  weight  of  an 
end-item  (e.g.,  a  satellite). 

B.  Model  Domain 

Figure  1  illustrates  the  interrelationships 
between  cost  estimation  uncertainty,  require¬ 
ments  uncertainty,  and  system  definition  uncer¬ 
tainty.  The  n-system  configurations  shown  are  in 
response  to  the  effects  of  requirements  uncer¬ 
tainty.  The  cost-schedule  probability  models 
described  in  this  paper  apply  within  the  domain 
shown  in  figure  1.  They  provide  probability- 
based  estimates  of  a  system's  cost  and  schedule 
for  a  given  system  configuration.  When  require¬ 
ments  uncertainty  necessitates  defining  an 
entirely  new  system  configuration,  new  cost- 
schedule  probability  models  must  also  be 
developed  specific  to  that  configuration. 

When  a  joint  cost-schedule  probability 
model  has  been  developed  on  a  program, 
analyses  can  be  conducted  to  support  risk 
management  decisions.  These  include: 

Baselining  Program  Cost  and  Schedule  Risk — For  a 
given  system  configuration,  acquisition  strategy, 
and  resource  estimation  approach  baseline 
probability  distributions  of  a  program's  cost 
and  schedule  can  be  developed.  These  distribu¬ 
tions  can  be  periodically  updated  as  the  pro¬ 
gram's  uncertainties  change  across  the 
acquisition  milestones.  Generating  these  distrib¬ 
utions  supports  defining  a  program  cost  and 
schedule  that  simultaneously  have  a  specified 
probability  of  not  being  exceeded.  This  distribu¬ 
tion  also  provides  program  managers  with  an 
assessment  of  the  likelihood  of  achieving  a  bud¬ 
geted  or  proposed  cost  and  schedule  or  cost  for 
a  given  schedule. 


I - Requirements  Uncertainty - 1 

System  •  •  •  System 

Configuration  1  Configuration  n 


Figure  1.  Types  of  Uncertainty  Captured  By 
Models 
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Estimating  Reserves — An  analytical  basis  for  ass- 
esing  cost  reserve  for  a  given  schedule,  or  set  of 
schedules,  as  a  function  of  the  uncertainties  spe¬ 
cific  to  a  given  system  configuration  can  be 
developed.  Sensitivity  analyses  can  be  conduct¬ 
ed  to  assess  how  reserve  levels  are  affected  by 
changes  in  specific  program  risks.  In  addition, 
the  relationship  between  the  amount  of  reserve 
to  recommend  for  a  given  level  of  confidence 
and  a  given  schedule  can  be  examined. 

Conducting  Risk  Mitigation  Tradeoff  Analyses — 
Models  can  be  developed  to  study  the  payoff  of 
implementing  specific  risk  mitigation  activities 
(e.g.,  prototyping)  on  reducing  cost  and  sched¬ 
ule  variances.  For  instance,  suppose  the  cost 
and  schedule  variances  of  a  program  are  driven 
by  uncertainty  in  the  amount  of  new  code  to 
develop.  Using  these  models,  it  can  be  deter¬ 
mined  whether  investing  in  a  prototyping  effort 
markedly  reduces  this  variance,  and,  therefore, 
lessen  the  cost  and  schedule  reserves  needed  for 
the  program. 


III.  BIVARIATE  PROBABILITY  MODELS 
FOR  COST  AND  SCHEDULE 

This  section  presents  two  bivariate  probabil¬ 
ity  models  for  quantifying  uncertainty  in  cost 
and  schedule  estimates.  Specifically,  the  bivariate 
lognormal  and  bivariate  normal-lognormal 
models  are  described.  They  are  candidate  theo¬ 
retical  distributions  that  might  be  assumed  by  an 
analyst  when  estimates  of  joint  or  conditional 
cost-schedule  probabilities  are  needed. 

These  models  have  two  properties  desirable 
for  cost  analysis  applications.  First,  they  directly 
incorporate  correlation  between  cost  and  sched¬ 
ule  on  a  given  program.  Second,  their  marginal 
distributions  are  either  both  lognormal  or  normal 
and  lognormal.  The  latter  property  is  reflective  of 
distributions  frequently  observed  in  Monte  Carlo 
simulations  of  program  cost  and  schedule  [4,5]. 
The  following  briefly  describes  each  model. 
Throughout  the  remainder  of  this  paper,  the  ran¬ 
dom  variables  and  X2  will  denote  program 
cost  and  schedule,  respectively. 


lognormal  distribution  frequently  approximates 
the  sum  of  many  positively  correlated  cost  ele¬ 
ment  random  variables.  Similarly,  program 
schedule  also  tends  toward  lognormality  if  it  is 
the  sum  of  many  positively  correlated  schedule 
activities.  Thus,  correlated  lognormals  often 
characterize  a  program's  cost-schedule  uncer¬ 
tainties.  In  cases  where 

•  univariate  lognormals  approximate  a 
program's  cost  and  schedule  distributions, 

•  circumstances  (e.g.,  time,  computing 
resources)  preclude  the  development  and  imple¬ 
mentation  of  formal  Monte  Carlo  simulations, 
then  the  bivariate  lognormal  can  serve  as  an 
assumed,  albeit  non-unique,  analytical  model  of 
the  joint  cost-schedule  distribution. 


A.I  Model  Definition 

Suppose  we  have  two  random  variables 
=  In  and  Y2  =  In  X2  where  and  X2  are 
defined  on  0  <  <  cxd  and  0  <  X2  <  ©o.  If  and 

Y2  each  have  a  normal  distribution  then,  for  i  =1, 
2,  the  mean  and  variance  of  Y-  are 


E(Yi)  =  ^Y^ 

Var(Y,)=aY^  =(jf  =  \n 


(/^X, 

(/^X, 


(1) 


(2) 


Assume  that  the  pair 

(X|,Y2)~  bivariate  LogN({^^\^2  )do-p(T2  ,Pj  ^  ))(3) 

is  a  bivariate  lognormal  distribution  with  density 
function 


fx,,)^(xi,X2)  = 


_ 1 

(27r)aia2-^U^xiX2 

0  otherwise 


(4) 


where 


A.  The  Bivariate  LogNormal  Model 

This  section  presents  the  bivariate  lognormal 
model  and  summarizes  its  major  characteristics. 
Monte  Carlo  simulations  suggest  that  the 
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and  /X-  and  cr2  (f  =  1,  2)  are  given  by  equations  , 

(1)  and  (2).  The  correlation  term  pj  j  equation 
(4)  is 


P].2  ~ 


1 


-In 


(5) 


2-^Pi.2  ) 


1 


(13) 


with 

z^  =  C7f(l-pf^2)^  ^  =  cr2(l-Pu) 


A  JJ  Model  Characteristics 


The  marginal  distributions  of  the  bivariate 
lognormal  distribution  are 


-/il 


(6) 


A.III  An  Illustration 

This  section  illustrates  an  application  of  the 
bivariate  lognormal  distribution.  Suppose  the 
software  cost  and  schedule  of  an  Ada 
development  task  are  needed.  A  simple  Ada 
cost-schedule  estimation  model  is  presented  in 
figure  2. 


and 


1  1*^2 


(7) 


^JlK  02X2 
The  conditional  distributions  are 


X^\x2-  LogNifii  +  — pj  2(lnx2  ”^2)  ’  ^?(i"P?.2)) 

(8) 

X2IX]  ~  LogN{^L2+—p\,2{\nxx  -Pl),U2(l-Pu)) 

^1  (9) 


where 


-^Pl  ^  U. - ^ 

£(X,|x2)  =  x|’2  ■  c 


„2Pl.2/l-^A,2/i2+T<7fd-P?,2) 


(10) 


£(X2|xi)  =  <'  e 


P1.2  h2~~^P\,2h'\  +1^2  (^"Pl, 2) 


(11) 


Var{Xi\x2)  = 


2-^Pi.2  2(Pi“'^Pi.2P2)^,2"^,,z^^ 


(6^^  -1) 

(12) 


Figure  2.  Ada  Cost-Schedule  Estimation  Model 

Suppose  the  uncertainty  in  the  amount  of 
Ada  source  lines  to  be  developed  is  given  as  a 
triangular  distribution,  whose  range  of  values  is 
shown  in  figure  3.  The  Ada  development  effort 
(in  staff  months)  is  estimated  according  to 

hjjor:  Vi.'KSource  !Ane^{  1  (14) 

where 

189, 0.277),  (15) 
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The  development  schedule  (in  months)  is 
estimated  according  to 

Schedule  =  4. Esrhed  (16) 

where 


0 .062 


Figure  3.  Ada  Source  Lines- 
Triangular  Distribution 

The  development  cost  is  estimated  by 

Cost  =  X  (Effort)  (18) 

where  A  =  $12,000  per  staff  month. 

A  Monte-Carlo  simulation  was  performed 
for  the  model  defined  by  equations  14-18. 
Summary  cost-schedule  statistics  generated  from 
that  simulation  are  provided  in  table  2. 

Table  2.  Software  Cost  and  Schedule  Statistics 

i  X\  fix  Ox  Ox 

t  I 


1  Cost($M)  9.8  24.01  4.9 

2  Schedule  (Mos)  32.6  90.25  9.5 


Correlation  PXj,Xo 

A  comparison  of  the  simulated  marginal 
distributions  of  Xj  and  X2  with  assumed  theoret¬ 
ical  lognormals  is  presented  in  figure  4.  Observe 
how  closely  the  simulated  distributions  match 
the  hypothesized  lognormal  distributions.  The 
simulation  provides  empirical  evidence  that 
the  Ada  cost  and  schedule  distributions,  in  this 


illustration,  can  be  characterized  by  correlated 
univariate  lognormals.  From  this  result,  it  is 
reasonable  to  assume  that  the  bivariate  lognor¬ 
mal  is  representative  of  the  joint  cost-schedule 
distribution.  However,  such  an  assumption  does 
not  guarantee  it  is  the  unique  distribution.  The 
true  joint  distribution  of  (XpX2)  cannot  be 
uniquely  determined  from  the  marginal  distribu¬ 
tions  of  X^  and  X2,  unless  they  are  uncorrelated. 
From  the  data  in  table  2,  the  parameters  needed 
to  specify  a  bivariate  lognormal  model  can  be 
computed.  These  data  are  summarized  in  table  3. 


Probability 


Figure  4.  Simulated  versus  Theoretical 
Marginal  Distributions^ 


Table  3.  Joint  Probability  Model  Parameter  Set 

2 

t  |Ii  Oi  Gi 

Bivariate  LogNormal  Model 

1  2.171  (Eqt  1)  0.223  (Eqt  2)  0.472 

2  3A44(Eqtl)  0.082  (Eqt  2)  0.286 

Correlation  ^  ^'48  (Eqt  5) 


Once  the  joint  distribution  of  (X^,X2)  has 
been  specified,  joint  cost-schedule  probabilities 
can  be  computed.  For  instance,  suppose  (X^,X2) 
is  assumed  to  be  bivariate  lognormal  with  para¬ 
meters  given  in  table  3.  Then  the  probability  that 
the  Ada  development  will  cost  between  7  and  14 
million  dollars  ($M)  and  be  completed  between 
24  and  36  months  (mos) 
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^The  simulated  marginal  distributions  are  shown  by  the  squares.  The 
assumed  marginals,  theoretical  lognormal  distributions,  are  shown  by 
the  solid  lines. 
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Pi$lM  <  X,  <  $14M  and  24  mos  <  X2  <  36  mos) 

r36  r]4 

J24J7  fx^.X2^^^’^2)dxidx2  =0.29 

In  this  case,  the  integrand  is  defined  by 
equation  4.  Joint  cost-schedule  probabilities  for 
other  regions  of  interest  are  determined  in  a  sim¬ 
ilar  manner. 

Cost  analyses  must  often  focus  on  assessing 
the  impact  that  a  set  of  schedule  outcomes  has 
on  the  likelihood  that  an  estimated  program  cost 
will  not  be  exceeded.  To  make  such  assessments 
conditional  distributions  are  needed.  For  the 
Ada  cost-schedule  model,  the  conditional  cost 
distribution  given  a  schedule  outcome  X2  is 

Xi|x2  ~  LogN(2.171  +  0.792(lnx2  -  3.444),  0. 172)  (19) 

Table  4  provides  the  mean  and  standard 
deviation  of  1 X2  given  two  values  of  ^2.  From 
figure  5,  the  impact  of  a  schedule  outcome  on  the 
probability  distribution  of  cost  can  be  seen.  As  X2 
increases  the  cumulative  conditional  cost  distrib¬ 
utions  become  'Tazier.'"  The  increased  laziness 
reflects  a  growth  of  nearly  3  million  dollars  in 
E(Xj  I X2)  if  the  development  schedule  is  3  years 
(36  months)  instead  of  2  years  (24  months). 
Although  the  higher  cost  may  be  undesirable,  it  is 
accompanied  by  a  3  year  schedule,  which  has  a 
lower  chance  of  overrun  (refer  to  figure  4)  than 
the  less  expensive  but  higher  risk  2  year  schedule. 

Linkages  such  as  these  between  cost  uncer¬ 
tainty  and  schedule  can  be  made  through  the  use 
of  conditional  distributions.  In  the  early  stages  of 
a  program,  conditional  distributions  provide 
decision-makers  valuable  insights  into  the  likeli¬ 
hood  of  achieving  cost  and  schedule  goals. 

Table  4.  Statistics  from  the  Conditional 
Distributions  in  Figure  5 

Given  JC2  E(X]\x2)  o{X^\x2) 

(Months)  ($M)  ($M) 

24  7.7  3.3 

36  10.7  4.6 


Figure  5.  A  Family  of  Conditional  LogNormal 
Cost  Distributions 
Ada  Cost-Schedule  Model  Example 

B.  A  Bivariate  Normal-LogNormal  Model 

This  section  presents  a  bivariate  normal-log¬ 
normal  model  and  summarizes  its  major  charac¬ 
teristics.  An  important  feature  of  this  model  is  its 
marginal  distributions.  One  marginal  is  normal 
and  the  other  is  lognormal. 

The  previous  section  described  circum¬ 
stances  that  give  rise  to  lognormal  distributions 
for  program  cost  and  schedule.  Normal  distribu¬ 
tions  also  occur  frequently.  For  instance,  when 
program 

•  cost  is  the  sum  of  many  uncorrelated  WBS 
cost  element  random  variables, 

•  schedule  is  the  sum  of  many  independent 
activities  in  a  network, 

then  the  normal  often  approximates  the  distribu¬ 
tion  of  program  cost  or  schedule. 

In  addition  to  the  above  instances,  a  normal 
distribution  for  program  cost  can  arise  from  the 
way  cost  element  distributions  interact  within  a 
WBS.  For  example,  when  the  following  condi¬ 
tions  exist 

•  the  PME  segment  (refer  to  table  1)  cost  is 
normally  distributed, 

•  the  PME  cost  variance  dominates  a 
program's  total  cost  variance, 

•  the  SS  segment  (refer  to  table  1)  is  highly 
correlated  to  a  normally  distributed  PME  cost. 
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then  program  cost  is  approximately  normal. 
Normality  can  also  result  when  the  SS  segment 
is  uncorrelated  to  a  normally  distributed 
PME  cost. 

In  summary,  when  normal  and  lognormal 
distributions  characterize  either  the  cost  or  the 
schedule  of  a  program,  the  bivariate  normal-log¬ 
normal  distribution  can  serve  as  an  assumed 
joint  probability  model.  This  assumption,  how¬ 
ever,  does  not  imply  that  the  bivariate  normal- 
lognormal  is  the  unique  distribution. 

B.I  Model  Definition 

Suppose  we  have  two  random  variables 
Y-y  =  Xj  and  Y2  =  /nX2  where  and  X2  are 
defined  on  —00  <  <  00  and  0  <  ^2  <  If 

and  Y2  each  have  a  normal  distribution  then,  for 
/  =1, 2,  the  mean  and  variance  of  Y-  are 

E(YO  =  liiY,  (20) 


P\,2  PY1J2  Px^JnX2  Px.,> 


BJl  Model  Characteristics 

A  characteristic  of  the  bivariate  normal-log- 
normal  distribution  is  that  the  distribution  of  X^ 
is  normal  and  the  distribution  of  X2  is  lognor¬ 
mal.  These  marginal  distributions  are  given  by 


CJi 


y  ZTT  O1X2 


The  conditional  probability  density  function 
of  X^  given  X2  =  X2  is 

Yi|x2  -  N{jl^  +— Pl,2(^^i-^‘2  -P2)’  ^?(1-Pu)  ) 


VariY^)  =  aY^=Gx,=(y\  (21) 

E{y2)  =  l-h\_  =;^2 

Var(Y2)  =  (Ty^  =  =  In  - - - ^ — -  (23) 

[  (Mx.  r 

Assume  that  the  pair 

(X|,X2)~  Bivariate  N U)gN{{^^,ii2),{(y\  ,a\,p^  ^)) 

(24) 

is  a  bivariate  normal-lognormal  distribution 
with  density  function 


similarly 

X2\x]  ~  Loi>N(ii2  +—pi,2U]  -M,),  (Tj(l -Ptp)  ) 

(30) 

The  conditional  means  and  variances  are 


£(Xi|^2)  =  Pi  +~Pi  2(^^^'2  “P2) 


E{X2\xx)  =  e 


1(2  +— Pi  ,2  (^1  -P 1 )+  7  <^2  0  ~Pk2  ) 


(31) 

(32) 


Var{X^\x2)  =  GU\-pl2) 


(33) 


and  jLt-  and  or2  (/  =  1,2)  are  given  by  equations 
(20)  through  (23).  The  correlation  term  2 
equation  (25)  is 


2(P2+^Pi  2(-^l-Pl)) 

Var(X2\xi)  =  e  - 1)  (34) 


IV.  Summary  Comments 

The  joint  probability  models  described  in 
this  paper  provide  an  analytical  basis  for  com¬ 
puting  joint  and  conditional  cost-schedule  prob¬ 
abilities.  These  models  are  particularly  useful 
when  (i)  quick  probability-based  assessments  of 
achieving  program  cost  and  schedule  are 
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needed,  (ii)  the  work  breakdown  structure  is 
used  as  the  primary  cost-schedule  estimation 
method,  and  (iii)  time  or  computing  resources 
preclude  the  use  of  Monte  Carlo  simulation. 

Selection  of  a  particular  model  is  guided  by 
the  marginal  distributions  it  produces.  For  exam¬ 
ple,  if  the  individual  distributions  of  and  X2 
are  normal  and  lognormal,  then  the  bivariate 
normal-lognormal  model  might  be  chosen  for 
the  joint  distribution  of  the  pair  (XpX2).  This  is 
because  the  bivariate  normal-lognormal  model 
produces  normal  and  lognormal  marginal  distri¬ 
butions.  However,  it  must  be  viewed  that  choos¬ 
ing  this  particular  model  makes  an  assumption 
about  how  the  pair  of  random  variables  (X|,X2) 
is  jointly  distributed.  Mentioned  previously,  the 
true  joint  distribution  of  (X^,X2)  cannot  be 
uniquely  determined  from  only  the  individual 
distributions  of  X^  and  X2. 

A  parameter  required  by  the  models  in  this 
paper  is  the  correlation  between  program  cost 
and  schedule.  This  is  a  difficult  parameter  to 
determine.  Ideally,  it  should  be  derived  from  val¬ 
ues  sampled  by  a  Monte  Carlo  simulation  of  the 
cost-schedule  relationships  established  in  a 
program's  work  breakdown  structure.  Subjective 
assessments  can  be  used  in  the  absence  of  a 
simulation. 

An  important  consideration  regarding  these 
models  is  that  they  do  not  reflect  the  causal 
impact  that  schedule  compression  or  extension 
has  on  cost.  Cost  and  schedule  are  treated  as  cor¬ 
related  random  variables  whose  range  of  values 
are  reflected  by  their  marginal  distributions. 
These  ranges  result  from  quantifying  the  uncer¬ 
tainties  associated  with  cost  estimation  uncer¬ 
tainty  and  system  definition  uncertainty. 
Unrealistically  compressing  or  extending  sched¬ 
ule  (due  to  missed  milestones  or  program  re¬ 
plans)  can  incur  increased  cost.  In  these 
circumstances  a  reassessment  of  the  system's 
cost-schedule  risk  is  warranted. 

In  an  environment  of  limited  funds  and 
increasingly  challenging  schedules,  it  is  incum¬ 
bent  upon  analysts  to  continually  examine 
affordability  concerns  relative  to  the  likelihood 
of  jointly  meeting  cost  and  schedule  and  cost  for 
a  given  feasible  schedule  against  specific  trade¬ 
offs  in  system  requirements,  acquisition  strate¬ 
gies,  and  post-development  support.  Enabling 
options  to  be  explored  that  offer  decision-makers 


economically  sound  and  risk  mitigating  choices 
throughout  the  life  of  a  program  is  the  cost  ana¬ 
lyst's  aim  and  opportunity.  Models  and  method¬ 
ologies  are  tools  that  provide  a  means  to  that 
end. 
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INTRODUCTION 

This  paper  considers  two  contributing 
causes  of  instabilities  in  combat  models  and 
complex  simulations.  These  are  (1)  computer 
arithmetic  and  (2)  nonlinear  effects,  includ¬ 
ing  chaos.  The  paradigm  of  dynamics  and 
dynamical  systems  sets  this  discussion.  A 
reason  for  this  is  that  many  computations 
are  performed  algorithmically.  When  a  com¬ 
putation  requires  algorithmic  iteration,  the 
algorithm  functions  as  a  dynamical  system 
which  may  exhibit  computational  chaos. 
Complex  models  are  decision  making 
models  that  are  designed  using  two  or  more 
different  interacting  modeling  paradigms. 
For  this  reason  combat  models  are  complex 
models.  The  design  of  combat  models  is 
based  on  at  least  two  paradigms:  discrete 
event  simulation  and  algorithmic  calculations 
using  computer  arithmetic.  Vagaries  of  com¬ 
puter  arithmetic  cause  phase  changes  and, 
therefore,  timing  problems,  in  computation¬ 
al  intensive  discrete  event  simulations  of 
decision  making  models.  Nonlinear  effects 
cause  chaos  and  structural  variance  in  the 
time  evolution  of  complex  models. 

Discussions  of  computer  arithmetic  are 
focused  on  instabilities  inherited  from  real 
arithmetic.  We  give  a  real  world  example  of 
propagation  of  timing  errors  that  are  a  direct 
effect  of  computer  arithmetic. 

Nonlinear  effects  are  explored  easily  in 
simulations  of  simple  models  of  attrition 
and  reinforcement.  We  give  a  several  para¬ 
meter  family  of  models  of  attrition  and  rein¬ 
forcement.  We  analyze  this  family  for  chaos, 
strange  attractors,  and  repellers  of  the 
dynamics. 

COMPUTER  ARITHMETIC  EFFECTS 

We  give  below  a  real  world  example  of 
computer  arithmetic  effects  and  their  tragic 
consequences.  This  demonstrates  the  fact 
that  arithmetic  effects  can  have  significant 
consequences.  For  simulations  these  effects 
need  not  be  limited  to  a  statistically  insignifi¬ 
cant  (less  than  5%)  range.  Predicting  the 
extent  of  divergence  introduced  by  vagaries 
of  computer  arithmetic  in  complex  simula¬ 
tions  is  difficult  because  of  the  enormous 
numbers  of  computational  paths.  In  the 
Patriot  example  we  see  there  is  only  one 
computational  path  of  significance. 


A  DETAILED  ANALYSIS  OF  THE 
PATRIOT  EXAMPLE 

A  timing  problem  is  illustrated  by  a  sig¬ 
nificant  real  world  example.  An  attack  by  a 
Scud  missile  on  Dhahran,  Saudi  Arabia,  dur¬ 
ing  the  Persian  Gulf  war,  was  not  engaged. 
As  documented  in  a  U.S.  General 
Accounting  Office  report,  an  accumulation 
of  timing  error  caused  the  failure  of  a  radar 
system  to  detect  and  to  identify  a  missile 
that  should  have  been  defended  against  by 
the  Patriot  [U.S.  GAO,  1992].  This,  in  turn, 
resulted  in  a  failure  to  launch  a  Patriot.  The 
timing  problem  was  a  mismatch  between  a 
timing  computation  and  the  stroboscopic 
view  by  the  radar.  Time  was  accumulated  as 
integer  numbers  of  tenths  of  a  second.  Thus, 
100  tenths  equals  10  seconds.  However,  the 
arithmetic  was  done  in  binary,  not  in  deci¬ 
mal.  If  finite  precision  is  used,  there  is 
always  a  rounding  error  in  the  binary  repre¬ 
sentation  of  1/10.  The  timing  error  that 
results  from  the  truncated  expansion  of  1/10 
is  directly  proportional  to  the  integer  num¬ 
ber  of  tenths.  When  the  system  operates  for 
more  than  a  few  hours  this  error  is  apprecia¬ 
ble.  In  this  particular  case,  the  system  oper¬ 
ated  for  100  hours  or  3,600,000  tenths  of  a 
second.  Computer  arithmetic  with  24  bits 
was  used  for  floating  point  calculations.  A  24 
bit  word  contains  a  sign  bit,  an  exponent 
and  a  significand  (also  called  mantissa). 
Precision  refers  to  the  number  of  bits  in  a 
significand.  Precision  20  results  in  an  error  of 
about  2”^^  in  the  binary  representation  of 
1/10.  One  tenth  is  written  as  a  binary  expan¬ 
sion  0.00011001100110011001100  1 1100... 
where  truncation  error  is  about  2"^^  at  preci¬ 
sion  20.  The  difference  between  1/10  and  the 
expansion  truncated  at  precision  20  is  exact¬ 
ly  0.8  X  2”^^.  The  floating  point  calculation  is 
in  error  by  3,600,000  X  0.8  X  2“^^, 
(=  0.343322..),  about  1/3  second.  This  error 
caused  a  shift  of  about  700  meters  in  the 
range  gate  (=  1  /3  sec  x  2000  m/ s,  the  termi¬ 
nal  velocity  of  Scud)  that  was  sufficient  for 
the  radar  detection  logic  to  fail  to  identify 
and  flag  the  incoming  ScuD  as  a  missile  that 
should  be  engaged  [Palmore,  1992a].  See 
table  on  page  49.  , 

COMPUTER  ARITHMETIC 

Computer  arithmetic  is  the  implementa¬ 
tion  in  hardware  of  finite  precision  real 
arithmetic.  Real  arithmetic  consists  of  the 
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field  of  real  numbers  with  the  binary  operations 
of  (+,  X  ).  The  field  axioms  are  associativity  and 
commutativity  in  addition  and  multiplication, 
distributivity  with  respect  to  addition  and  multi¬ 
plication,  existence  of  additive  and  multiplica¬ 
tive  inverses,  and  identities  for  addition  and 
multiplication.  Multiplication  is  iterative  addition  of 
two  real  numbers  and  division  is  iterative  subtraction 
of  a  nonzero  real  number  from  a  real  number.  These 
operations  need  not  terminate  in  finitely  many 
steps  for  arbitrary  computable  real  numbers 
because  arbitrary  computable  reals  have  infinite 
digit  strings.  Instabilities  of  real  arithmetic 
are  inherited  by  finite  precision  computer 
arithmetic. 

MODELS  OF  COMPUTER  ARITHMETIC 

It  is  vital  to  recognize  the  importance  of 
computer  arithmetic  and  its  vagaries  in  the  exe¬ 
cution  of  simulations  of  complex  models  where 
unstable  behavior  in  embedded  dynamical  sys¬ 
tems,  even  the  arithmetic  itself,  can  cause  exces¬ 
sive  divergences  in  results  [Palmore,  1992c, 
1991a,b;  Palmore  and  Herring,  1990]. 

Models  of  computer  arithmetic  include  algo¬ 
rithms  that  are  used  to  implement  the  operations 
of  addition,  multiplication,  subtraction,  and  divi¬ 
sion.  These  latter  algorithms  are  implemented 
usually  in  hardware  within  the  numeric 
coprocessor.  Although  the  fundamental  elements 
of  shift  and  add  are  employed,  there  is  a  choice 
in  the  algorithm  used  for  division.  Models  of 
computer  arithmetic  depend  upon  base  of  arith¬ 
metic,  data  types,  precisions,  roundings,  and 
specifications. 

The  IEEE  standards  for  binary  floating  point 
arithmetic  and  radix  independent  floating  point 
arithmetic  allow  12  combinations  of  precisions 
and  roundings  [ANSI/IEEE  1985].  IEEE  preci¬ 
sions  are  24,  53,  and  64  binary  bits.  These  are 
single,  double  and  extended  precisions,  respec¬ 
tively.  IEEE  roundings  are  round  to  nearest,  and 
three  directed  roundings:  truncate  (chop  or 
round  toward  0),  round  up  (toward  +oo),  round 
down  (toward  —  ©o). 

Each  combination  of  precision  and  rounding 
is  a  different  model  of  computer  arithmetic  for 
the  computation  of  numerical  quantities. 
These  12  combinations  of  precision  and  round¬ 
ing  allow  a  '"poor  man's"  test  of  sensitivity.  By 


running  a  computer  simulation  using  different 
combinations  of  precision  and  rounding,  a 
designer  can  test  its  sensitivity  to  computer 
arithmetic. 

These  models  of  computer  arithmetic  make 
a  difference.  For  example,  when  using  a  pseudo 
random  number  generator  of  the  form  x  a  •  x 
mod  m  (a  PRIME  MODULUS  MULTIPLICA¬ 
TIVE  LINEAR  CONGRUENTIAL  GENERA¬ 
TOR)  with  a  =  7  ^  (=  16807)  and  m  =  2  ~  1 

(=  2147483647)  at  least  46  bits  of  precision  are 
needed  to  contain  intermediate  numerical 
results.  Consistent  answers  cannot  be  obtained  if 
this  requirement  is  not  met.  The  reason  for  this  is 
that  when  rounding  occurs  in  the  integer  arith¬ 
metic  (in  the  floating  point  environment  where 
less  than  46  bits  are  allowed)  the  multiplying  fac¬ 
tor  of  16807  causes  immediate  divergence 
between  the  exact  PRNG  sequence  and  the 
computed  sequence  [Palmore,  1992b,  1991a,b]. 

Consider  the  Simscript  11.5  pseudo  random 
number  generator.  The  multiplier  a  =  630360016 
and  prime  m  =  2  -  1  are  the  parameters 

[Fishman  and  Moore,  1986].  Intermediate  prod¬ 
ucts  of  multiplication  exceed  double  precision. 
The  code  uses  partial  products 

(A  *  2^  +  B)  •  (C  •  2^  +  D)-A  •  C  •  2^^  + 

(A  •  D  +  B  •  C)  •  2^  +  B  •  D. 

The  Simscript  11.5  generator  fits  existing 
double  precision  by  computing  and  retaining 
partial  products  A  •  C,  A  •  D,  B  •  C,  and  B  •  D. 
Special  care  has  been  taken  to  implement  this 
simple  PRNG  so  that  there  is  1  -  1  correspon¬ 
dence  between  the  discrete  arithmetic  model  and 
its  computer  implementation. 

INSTABILITY  OF  MULTIPLICATION 
AND  DIVISION 

As  every  school  child  knows,  multiplication 
and  division  are  hard  to  do  correctly  by  hand  because 
once  a  mistake  is  made  the  rest  of  the  answer  is 
wrong. 

Obviously,  a  division  computation  diverges 
rapidly  once  a  mistake  is  made.  Division  is 
unstable  because  division  is  equivalent  to  a 
Bernoulli  shift,  a  known  chaotic  operation 
[Palmore  and  McCauley,  1987;  Palmore,  1988; 
Palmore  and  Herring,  1990]. 
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EFFECTS  OF  TINY  ROUNDINGS  ON 
SIMULATIONS 

If  computer  simulations  are  computational 
intensive,  then  the  computations,  performed 
typically  by  algorithms  as  iterative  operations, 
can  inherit  instabilities  from  computer  arith¬ 
metic.  The  important  point  is  this.  The  reason 
algorithms  perform  calculations  efficiently  is  that 
a  lot  of  information  is  gained  at  each  step.  For 
division  at  least  one  digit  is  gained  at  every  step. 
Very  efficient  division  algorithms,  such  as 
Newton's  method,  allow  a  geometrically  increas¬ 
ing  number  of  digits  to  be  gained  step  by  step. 
Chaotic  processes  are  being  used  for  this  pur¬ 
pose.  If  small  errors  are  found  in  an  unstable  cal¬ 
culation,  then  a  rapid  divergence  of  solutions  is 
guaranteed. 

THERE  IS  A  CURE  FOR  THE 
PROPAGATION  OF  ERRORS 

A  cure  for  the  propagation  of  errors  is  to  use 
indefinite  precision  calculations  so  that  small 
errors  remain  small.  It  is  when  a  calculation  hits 
a  boundary  that  an  effect  arises  that  is  similar  to 
turbulence  in  fluids.  In  this  case  finite  precision 
establishes  a  "boundary  layer"  in  which  turbu¬ 
lence  is  produced  and  extends  to  chaos  in  the 
"bulk  calculation." 

When  indefinite  precision  is  used,  the  num¬ 
ber  of  bits  in  a  calculation  is  allowed  to  grow  to  a 
much  larger  number  than  standard  precisions 
allow.  Bulk  effects  are  seen  only  when  a  bound¬ 
ary  is  reached  and  back  propagation  occurs.  This 
approach  is  effective  when  an  attempt  is  made  to 
hold  accuracy  to  a  few  digits  uniformly  through¬ 
out  a  calculation.  Thus,  one  ensures  enough 
room  to  work  so  that  the  number  of  steps  avail¬ 
able  for  errors  to  propagate  far  enough  in  preci¬ 
sion  to  reach  a  boundary,  to  be  reflected,  and  to 
propagate  back  to  affect  the  accuracy  desired  is 
greater  than  the  number  of  steps  required  to 
complete  a  calculation. 


AVAILABILITY  OF  INDEFINITE 
PRECISION  ARITHMETIC  FOR  EXACT 
COMPUTATIONS 

Indefinite  precision  is  not  available  in  high 
level  discrete  event  simulation  languages. 
Indefinite  precision  has  to  be  built  especially  for 
particular  calculations.  It  is  part  of  the  design  of 
algorithms  and  their  implementations  on 
machines. 

There  are  computational  programs  that  do 
indefinite  precision  arithmetic.  This  includes 
rational  arithmetic  where  answers  are  retained  in 
rational  type.  These  programs  give  exact 
answers. 

INFORMATION  LOSS  IN  COMPUTER 
ARITHMETIC 

Information  is  lost  in  finite  precision  arith¬ 
metic.  Consider  the  set  of  significands  in  preci¬ 
sion  N  that  represent  numbers  in  the  interval  1  < 
X  <  2.  What  is  the  subset  of  significands  of  num¬ 
bers  such  that  the  reciprocal  of  each,  expanded 
in  base  2,  has  period  not  exceeding  N?  Each 
rational  number  has  a  binary  expansion  that  is 
eventually  periodic.  Most  of  the  binary  expan¬ 
sions  of  reciprocals  have  periods  that  exceed  N. 
Thus,  finite  precision  does  not  hold  the  informa¬ 
tion  necessary  to  represent  the  period  of  the 
binary  expansions  of  most  numbers.  These 
defects  may  appear  to  be  highly  technical  in  ori¬ 
gin.  They  are  fundamental  in  the  sense  that 
changes  in  precision  and  rounding  do  not 
change  their  effects. 

EFFECTS  OF  COMPUTER  ARITHMETIC 
ON  ITERATIVE  PROCESSES 

Problems  with  computer  arithmetic  can 
arise  in  iterative  processes  such  as  discrete 
dynamical  systems.  An  example  is  given  by  the 
difference  equations  that  are  written  down  from 
the  continuous  system  of  Lanchestrian  differen¬ 
tial  equations.  These  differential  equations  have 
hyperbolas  as  solution  curves.  A  significant 
point  is  to  model  the  reinforcement  strategy 
easily.  As  in  every  discrete  dynamical  system  in 
which  the  model  is  a  set  of  difference  equations 
to  be  iterated  with  uniform  step  size,  the  time  is 
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measured  by  the  number  of  iterations  per¬ 
formed.  In  particular,  roundings  and  precisions 
will  affect  outcomes  as  iteration  proceeds.  Here 
is  the  main  question.  When  will  the  effects  of 
computer  arithmetic  be  observed?  The  results  of 
iterating  the  discrete  system  can  be  compared 
directly  with  evaluations  of  the  analytic  solution. 
When  an  attractor  of  the  dynamics  exists  the 
only  effect  likely  to  be  observed  is  a  change  in 
phase  of  outcomes  as  the  model  of  arithmetic  is 
changed.  With  limited  numbers  of  reinforcement 
blocks,  there  may  be  too  little  time  (that  is,  num¬ 
bers  of  steps)  for  computer  arithmetic  effects  to 
accumulate.  For  chaotic  behavior,  with  unlimited 
numbers  of  reinforcement  blocks  and  unlimited 
numbers  of  iterations,  it  is  a  question  of  how 
long  it  takes  before  significant  phase  changes 
between  the  outcomes  occur  using  two  different 
models  of  arithmetic.  One  measure  of  effect  is 
changing  the  win-loss  picture.  How  many  steps 
will  this  take?  [Palmore,  1991a,c] 

MATHEMATICAL  ANALYSIS  OF 
MODELS  OF  ATTRITION  AND 
REINFORCEMENT 

This  section  is  divided  into  two  parts.  The 
first  part  is  about  chaos;  the  second  part  is  about 
nonmonotonicity  of  battle  outcomes.  The  interest 
in  chaos  and  nonmonotonicity  of  simulated  bat¬ 
tle  outcomes  in  simple  models  of  attrition  and 
reinforcement  has  come  primarily  from  the  work 
of  Dewar,  Gillogly,  and  Juncosa  at  RAND 
(Dewar,  et.al.,  1991). 

A  mathematical  analysis  of  a  generic  model 
of  attrition  and  reinforcement  by  the  author 
below  demonstrates  chaos  as  a  consequence  of  a 
positive  Lyapunov  exponent  and  disorder  given 
by  moduli  operations  that  simulate  reinforce¬ 
ment  (Palmore,  1992a).  Unpredictability  in  actual 
battle  outcomes  is  suggested  by  Singleton's 
work  (Palmore,  1992a). 

We  show  how  nonmonotonicity  in  battle 
outcomes  is  obtained  for  free  by  the  addition  of 
"area  fire"  terms  to  linear  Lanchester  equations. 
We  prove  that  chaos  exists  both  in  the  linear  dis¬ 
crete  Lanchester  equations  with  reinforcement 
and  in  the  nonlinear  discrete  Lanchester 
equations  with  reinforcement. 


CHAOS  IN  MODELS  OF  AHRITION 
AND  REINFORCEMENT 

We  consider  the  following  system  of  first 
order  linear  differential  equations  of  Lanchester, 

dxidi  =  "  ay,  dy/dt  =  -  fe  (1) 

where  x,  y  >  0,  and  auxiliary  conditions,  a)  if 
y/x  >  r,  then  x  is  replaced  by  x  +  c,  repeatedly  N 

>  1  times,  until  y/ (x+Nc)  <  r  holds,  b)  if  y/x  < 
s,  then  y  is  replaced  by  y  +  d,  repeatedly  K  ^  1 
times,  until  (y+Kd)/x  >  s  holds,  where  a,b,c,d,r,s 

>  0,  and  r  >  s  >  V  (b/a). 

This  model  has  area  preserving  flow  because 
the  divergence  of  the  right  hand  side  equals  0.  It 
has  a  first  integral  I(x,y)  =  ay^  —  bx^.  The  solu¬ 
tion  curves  of  the  differential  equations  are 
hyperbolas  and  the  solutions  of  the  system  of 
equations  together  with  the  auxiliary  conditions 
are  arcs  of  hyperbolas  and  isolated  points. 

A  DISCRETE  MODEL  DERIVED  FROM 
THE  LINEAR  MODEL 

By  writing  down  a  set  of  first  order  linear 
difference  equations  to  replace  the  system  of  first 
order  linear  differential  equations  (1)  above,  we 
obtain  a  discrete  time  dynamical  systems  model 
of  attrition  and  reinforcement.  The  system  of 
linear  difference  equations  is, 

Vn+l  =  Vn  -  (2) 

where  x^^,  y^^  >  0,  for  all  n  ^  0,  h  >  0  is  a  parame¬ 
ter,  and  auxiliary  conditions,  a)  if  y^/ x^  >  r,  then 
is  replaced  by  x^  +  c,  repeatedly  N  ^  1  times, 
until  yj^/(Xj^+Nc)  <  r  holds,  and  b)  if  y^^/x^  <  s, 
then  y^^  is  replaced  by  y^^  +  d,  repeatedly  K  >  1 
times,  until  {y^+Kd)/x^  >  s  holds,  where 
a,b,c,d,r,s,  and  h  >  0,  and  r  >  s  >  V  (b/ a). 

For  h  9^  0,  the  flow  of  the  discrete  model  is 
area  decreasing  (the  determinant  of  the  mapping 
equals  1  -  h^ab  <  1.)  We  consider  only  those 
values  of  h  for  which  the  determinant  is  positive, 
i.e.  1  -  h^ab  >  0.  An  attractor  of  the  discrete  flow 
is  a  compact,  invariant  set.  Therefore,  any 
attractor  has  measure  0. 

Lyapunov  multipliers  and  Lyapunov  expo¬ 
nents  are  computed  easily.  The  multipliers  are 
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given  by  A  +  (h)  =  1  ±  hV(ab).  Lyapunov  expo¬ 
nents  are  A  +  (h)  =  In  [1  ±  hV(ab)].  By  scaling 
arguments  for  piecewise  linear  maps,  this  allows 
us  to  compute  the  Hausdorff  dimension  of  the 
attractor  A(h)  as 

HD[A(/z)]  =  l+X  +  (/z)/l\_(/z)l 

for  any  h  >  0.  For  h  >  0,  we  have  HD[A(h)]  =  2 

-  hV(ab)  +  O(h^ab).  Thus,  the  Hausdorff  dimen¬ 
sion  of  A(h)  is  less  than  2  for  h  >  0  so  that  A  is  a 
fractal  attractor. 

We  have  proved  the  existence  of  a  fractal 
attractor,  a  strange  attractor,  of  the  discrete  time 
dynamics  for  every  h  >  0  such  that  the 
determinant  of  the  mapping  is  positive. 

A  first  integral  for  this  discrete  flow  is  given 
by  Yn+V  V  yn>  =  ^^yn+l^  “  bx„^,2)/(ay^2 

-  bXj^^)  =  1  -  h^ab.  This  expression  is  indepen¬ 
dent  of  the  orbit.  The  integral  Kx^^^^  y^^) 

is  called  an  isoenergetic  first  integral  and  it 
depends  only  upon  h,  a,  and  b. 

It  is  clear  that  chaos  exists  in  this  dynamical 
system.  There  is  a  strange  attractor  and  the  sys¬ 
tem  has  a  positive  Lyapunov  exponent  for  every 
h  >  0  that  we  consider.  This  suffices  for  chaos 
(Palmore,  1991a).  The  positive  Lyapunov  expo¬ 
nent  ensures  a  sensitive  dependence  on  initial 
conditions  for  this  discrete  dynamical  system. 
Chaos  exists  for  all  choices  of  h,  a,  and  b  such  that 
1-h^ab  >  0.  In  particular,  the  Hausdorff  dimen¬ 
sion  of  the  attractor,  HD[A],  can  be  computed 
easily  as  a  function  of  the  parameters  h,  a,  and  b. 

The  cases  considered  by  Dewar,  Gillogly  and 
Juncosa  of  RAND  are  subsumed  by  the  family  of 
discrete  dynamical  systems  above  (Dewar,  et.  al., 
1991).  In  RAND's  simple  model  of  attrition  and 
reinforcement  the  discrete  dynamical  system  is 
piecewise  linear  with  a  constant  derivative. 
From  our  analysis  above  we  know  immediately 
several  properties  of  the  dynamics.  We  know  the 
Lyapunov  multipliers,  a  positive  Lyapunov 
exponent,  the  directions  of  expansion  and  con¬ 
traction,  the  measure  and  dimension  of  the 
attractor.  The  dynamical  system  is  chaotic  by  a 
positive  Lyapunov  exponent.  This  simple  model 
of  attrition  and  reinforcement  is  an  example  of 
predator-predator  interaction  with  restocking  of 
each  predator.  Combat  is  distinguished  from 
other  model  types:  predator-prey,  competing 
species,  and  logistic. 


NONMONOTONICITYAND 
NONLINEARITY  IN  MODELS  OF 
AHRITION  AND  REINFORCEMENT 

We  add  to  the  model  of  linear  differential 
equations  the  terms  for  "area  fire"  used  by 
Lanchester.  The  differential  equations  are 
written, 

dx/dt  =  -  Ay  -  ACxy,  dy/dt  =  -  Bx  -  BDxy 

(3) 

where  x,  y  >  0,  and  auxiliary  conditions,  a)  if 
y /x  >  r,  then  x  is  replaced  by  x  +  c,  repeatedly  N 
>  1  times,  until  y/ (x+Nc)  <  r  holds,  and  b)  if 
y/x  <  s,  then  y  is  replaced  by  y  +  d,  repeatedly 
K  >  1  times,  until  (y+Kd)/x  >  s  holds,  where 
A,B,C,D,c,d,r,s  >  0,  and  r  >  s  >  a/(B/A). 

We  remark  that  the  criterion  V(B/ A)  follows 
from  the  an  analysis  of  the  integral  curve  as  it 
approaches  the  origin.  This  model  has  area 
decreasing  flow  (the  divergence  of  the  right 
hand  side  equals  -  ACy  -  BDx  <  0  for  all  x,  y  > 
0.  This  implies  that  any  attractor  of  the  dynamics 
has  measure  zero.  Thus,  a  strange  attractor  exists 
when  the  flow  is  wrapped  around  a  bounded 
region  indefinitely  by  using  unlimited 
reinforcements. 

A  FIRST  INTEGRAL  (CONSTANT  OF 
MOTION) 

A  first  integral  (constant  of  motion)  for  the 
system  of  linear  differential  equations  is  Io(x,y)  = 
ay2  _  curve  defined  by  lQ(x,y)=C  (con¬ 

stant)  is  an  orbit  of  a  solution  of  the  differential 
equation.  The  orbit  through  (0,0)  is  a  straight  line 
y  =  V(b/a)  X,  X  >  0.  By  integrating  equations  (3) 
we  find  an  analytic  first  integral  I(x,y)  for  the 
nonlinear  differential  equations, 

I(x,y)  =  (B/C)x  -  (A/D)y  ™  {B/C^)lniUCx)  + 
(A/D2)/n(l+Dy). 

If  Cx  =  Dy  and  (B/C)x  =  (A/D)y,  then  the 
orbit  that  passes  through  the  origin  is  a  straight 
line.  In  any  event,  setting  I(x,y)  =  0  and  finding 
the  orbit  for  x,  y  >  0  yields  a  curve  that  separates 
the  quadrant  into  two  regions  where  a  win 
occurs  without  reinforcement.  In  order  to  specify 


Military  Operations  Research,  Spring  1996 


Page  49 


DYNAMICAL  INSTABILITY  IN  COMBAT  MODELS 


the  reinforcement  criterion  above,  that  r  >  s  > 
V(B/A),  we  examine  the  direction  of  approach  to 
the  origin  (0,0)  of  the  integral  curve  defined  by 
I(x,y)  =  0  for  X,  y  >  0.  We  want  to  find  the  tan¬ 
gent  direction  of  the  curve  at  (0,0)  that  is  the  lim¬ 
iting  direction  of  the  vector  field  (3)  as  the  curve 
approaches  (0,0).  The  tangent  direction  is, 

dy/dx  =  Bx{l+Dy)/[Ay(l+Cx)l  (4) 

A  simple  procedure  yields  the  limiting  direction 
along  the  curve  defined  by  I(x,y)  =  0.  We  set  y  = 
ax  and  substitute  into  (4).  This  yields  at  x  =  0  the 
expression  B/A  =  a^.  With  limited  reinforce¬ 
ments,  nonmonotonicity  of  battle  outcomes 
is  possible  in  this  model.  With  unlimited 
reinforcements  chaos  occurs. 

A  DISCRETE  MODEL  DERIVED  FROM 
THE  NONLINEAR  MODEL 

By  using  the  same  technique  to  write  down 
a  replacement  of  the  derivative  as  a  finite  differ¬ 
ence,  we  obtain  a  discrete  dynamical  system  for 
the  nonlinear  differential  equations  (3). 

\+i  =  \  -hAy„{l+Cx^), 

y«+i  ® 

This  system  of  difference  equations  is  non¬ 
linear  in  x^  and  y^^.  By  adding  criteria  for  rein¬ 
forcements  the  system  of  difference  equations 
and  auxiliary  conditions  is  nonlinear  in  two  dis¬ 
tinct  ways.  The  reinforcement  strategy  causes 
wraparound,  a  nonlinear  effect. 

CHAOS  AND  NONMONOTONICITY 

We  define  a  mapping  F  in  two  dimensions 

by, 

F:  (x,  y)  (x  -  /zAy[l  +Cx],  y  -  hBxU  +Dy])  (6) 

By  differentiating  (6)  we  obtain  the  Jacobian 
matrix  DF(x,y).  Finally,  by  computing  the  deter¬ 
minant  I  DF(x,  y)  I  we  obtain  the  Jacobian  J(F)  of 
the  mapping  F. 

J{F)  -  1-h^AB  -  (hBD+h^ABOx 

-  (hAC+h^ABD)y,  (7) 


For  X,  y  >  0,  J(F)  <  1.  This  shows  that  the 
nonlinear  ''area  fire"  terms  in  the  difference 
equations  (5)  only  increase  the  rate  of  area 
decrease  for  h  >  0.  Thus,  for  all  cases  in  which  h, 
X,  y  >  0,  the  measure  of  an  attractor  equals  0. 

We  show  by  using  (7)  that  a  positive 
Lyapunov  exponent  exists  for  bounded  orbits  of 
the  nonlinear  difference  equations  (5).  Let  A  + 
(x,y)  denote  eigenvalues  of  the  derivative  matrix 
of  (6).  We  compute  the  multipliers  from 

A2  -  [2  -  hiACy+BDx)]AF[l  - 
HACyFBDx)  -  h^AB(lFCxFDy)]^0. 

For  A,  B,  C,  D,  h,  x,  y  >  0,  we  obtain  for  A 
the  expression, 

A  =  1  +  ]4h(ACy+BDx)l  -  1  ±  V  (1  + 
[4AB(1  +Cx+Dy)/(ACy+BDx)2])]. 

This  demonstrates  that  A  ^  >  1  and  ^  <  1 
for  all  choices  of  A,  B,  C,  D,  h,  x,  y  >  0.  For 
C=D=0  we  recover  from  the  quadratic  equation, 
A  +  =  1  ±  hV(AB),  Lyapunov  multipliers  of  the 
linear  system. 

This  demonstrates  a  positive  Lyapunov 
exponent  for  every  orbit  of  the  system  (5)  that  is 
bounded  away  from  (0,0).  This  implies  chaos 
exists  in  the  discrete  dynamical  system  (5). 

Fig.  1  is  a  3-dimensional  Battle  Outcome 
Diagram  for  initial  strengths  of  Red  (fixed  per 
panel)  and  Blue  (increasing  along  horizontal). 
Fig.l  shows  nonmonotonicity  as  the  effect  of 
nonlinearity  as  the  strengths  of  area  fire  terms 
are  increased  from  zero  to  positive  in  0.01  incre¬ 
ments  of  the  coefficients,  A  nonlinearity  effect  is 
expected.  As  the  initial  strength  of  Blue  force  is 
increased,  we  expect  area  fire  to  be  significant. 
This  is  illustrated  in  Fig.  1  for  fixed  initial 
strength  of  Red.  We  follow  Blue's  initial  strength, 
moving  from  left  to  right  along  a  horizontal  line. 
At  first  Red  wins.  As  Blue's  initial  strength 
increases.  Blue  wins.  Eventually,  area  fire  gives 
Red  a  significant  advantage.  As  Blue's  strength 
increases  further.  Red  wins. 

CONCLUSIONS 

Mathematical  analysis  has  shown  the  exis¬ 
tence  of  chaos  for  the  system  of  linear  difference 
equations  with  reinforcements  and  the  existence 
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BATTLE  OUTCOME  DIAGRAM:  INITIAL  STRENGTH  OF  BLUE  VS.  NONLINEARITY 
(RED  INITIAL  STRENGTH  IS  FIXED  PER  PANEL) 

X  =  RED  WIN,  •  =  BLUE  WIN 
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Figure  1:  Battle  Outcome  Diagram:  Initial  Strength  of  Blue  vs.  Nonlinearity 


TABLE 


HOURS 

SECONDS^ 

CALCULATED  TIME 
(SECONDS) 

IN  ACCURACY 
(SECONDS) 

SHIFT 

(METERS) 

0 

0 

0 

.0 

0 

1 

3600 

3599.9966 

.0034 

7 

8 

28800 

28799.9725 

.0275 

55 

20 

72000 

71999.9313 

.0687 

137 

48 

172800 

172799.8352 

.1648 

330 

72 

259200 

259199.7528 

.2472 

494 

100 

360000 

*  359999.6667 

.3433 

687 

DATA  FROM  APPENDIX  II 

EFFECT  OF  EXTENDED  RUN  TIME  ON  PATRIOT  OPERATION 

GAO/IMTEC-92-26  Patriot  Missile  Software  Problem 

^  This  column  is  calculated  exactly  as  10  X  Integer  count  of  1/10  seconds. 

The  second  digit  6  is  a  misprint  in  the  report. 
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of  nonmonotonicities  for  the  system  of  nonlinear 
difference  equations  with  moduli  operations. 
Nonlinearities  in  the  equations  give  rise  to  non¬ 
monotonicity  and  the  reinforcements  with  posi¬ 
tive  Lyapunov  exponent  give  rise  to  chaos. 
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ABSTRACT 

The  United  States  Army  trains  thou¬ 
sands  of  new  soldiers  each  year  to  fill  vacan¬ 
cies  in  Army  organizations.  Initial  entry 
training  consists  of  two  sequential  phases: 
Basic  Combat  Training  followed  by 
Advanced  Individual  Training.  Until  re¬ 
cently  manual  heuristic  methods  were  used 
to  schedule  hundreds  of  training  companies 
for  initial  entry  training.  Scheduling  training 
companies  primarily  involves  deciding  how 
many  trainees  to  assign  to  training  compa¬ 
nies  each  week.  This  paper  formulates  a 
mathematical  dynamic  model  of  the  Basic 
Combat  Training  phase  of  initial  entry  train¬ 
ing.  We  also  present  two  approaches  for 
scheduling  training  resources.  One  is  a 
mathematical  decision  model  for  optimally 
scheduling  training  resources  based  on 
dynamic  programming.  The  second  is  a 
fully  automated  heuristic  procedure  imple¬ 
mented  in  an  operational  decision  support 
system  (DSS)  for  managing  the  Army's 
resources  for  initial  entry  training. 
Computational  experiments  reveal  that  the 
heuristic  procedure  developed  is  indeed 
computationally  efficient.  Furthermore,  the 
heuristic  provides  "good"  solutions  in  terms 
of  three  performance  measures:  training 


quality  as  measured  by  an  instructor-to- 
trainee  ratio,  resource  utilization,  and 
training  costs. 

1.  INTRODUCTION 

Each  year  the  United  States  Army 
recruits  and  trains  thousands  of  new  sol¬ 
diers.  Responsibility  for  initial  entry  training 
belongs  to  the  Unites  States  Army  Training 
and  Doctrine  Command  (TRADOC)  head¬ 
quartered  at  Fort  Monroe,  Virginia.  The  mili¬ 
tary  installations  responsible  for  training 
new  soldiers  are  scattered  across  the  United 
States  as  shown  below  in  Figure  1. 

Entry  level  training  consists  of  two 
phases:  Basic  Combat  Training  (BCT),  nor¬ 
mally  lasting  8  weeks,  followed  by 
Advanced  Individual  Training  (AIT)  that 
varies  from  5  to  50  weeks.  The  variability  in 
time  of  AIT  reflects  curriculum  differences 
across  the  AIT  programs  administered  at  dif¬ 
ferent  installations.  In  One-Station-Unit 
Training  (OSUT),  BCT  and  AIT  take  place  at 
the  same  installation.  Figure  2  illustrates  an 
aggregated  view  of  the  initial  entry  training 
process.  In  this  case,  the  training  companies 
from  all  installations  may  be  viewed  as  a 
single  group  of  reusable  resources. 


Figure  2:  Aggregated  view  of  the  initial  entry  training  process. 
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Proper  management  of  the  Army's  initial 
entry  training  program  demands  timely  schedul¬ 
ing  of  many  reusable  training  resources.  This 
paper  deals  with  the  complex,  practical  logistics 
problem  of  scheduling  hundreds  of  basic  train¬ 
ing  companies  throughout  the  planning  horizon 
for  training  new  soldiers.  Training  company 
scheduling  involves  deciding  how  many  trainees 
to  assign  to  training  companies  each  week  and 
how  many  weeks  the  companies  remain  busy 
with  training. 

Until  recently,  training  managers  used  man¬ 
ual  and  partially  automated  heuristic  methods  to 
schedule  training  companies  for  basic  training. 
Severe  shortcomings  exist  with  these  methods. 
First,  assigning  trainees  to  training  companies 
and  determining  the  number  of  weeks  a  training 
company  remains  busy  training  recruits  were 
decided  by  trial-and-error.  Second,  it  was  possi¬ 
ble  for  different  analysts  to  generate  different 
solutions  for  the  same  recruitment  scenario. 
Third,  no  methods  existed  for  conducting  com¬ 
parative  analyses  to  evaluate  the  quality  of  com¬ 
peting  feasible  training  schedules.  Finally,  the 
temporal  interdependence  of  decisions  makes 
decision  variables  in  the  future  periods  depend 
on  current  decision  variables.  This  complicates 
resource  scheduling  and  made  the  manual  gen¬ 
eration  of  week-by-week  training  schedules  an 
extremely  tedious,  time-consuming  task. 

The  major  objectives  of  this  paper  are  as  fol¬ 
lows.  First,  we  formulate  a  mathematical 
dynamic  model  of  the  Basic  Combat  Training 
phase  of  initial  entry  training.  Then  we  formu¬ 
late  a  decision  model  for  optimally  scheduling 
training  resources  based  on  dynamic  program¬ 
ming.  Next,  an  improved,  automated  heuristic 
procedure  for  scheduling  training  resources  is 
presented  that  we  implemented  in  a  decision 
support  system  (DSS).  The  heuristic  procedure 
incorporates  a  measure  of  training  "quality"  into 
the  objective  function  for  comparing  competing 
feasible  training  schedules.  Computational 
experiments  reveal  that  the  heuristic  procedure 
developed  is  computationally  efficient  and 
provides  "good"  scheduling  solutions.  Training 
schedule  quality  is  determined  by  three  per¬ 
formance  measures:  training  quality  as  mea¬ 
sured  by  an  instructor-to-trainee  ratio,  resource 
utilization,  and  training  costs. 


Previous  work  somewhat  related  to  our 
problem  includes  that  of  Yang  and  Ignizio  (1987). 
Their  problem  involved  scheduling  training 
resources  and  training  activities  for  a  fixed  num¬ 
ber  of  US  Army  battalions  located  at  the  same 
training  installation.  Constraints  existed  on  the 
type  and  quantity  of  training  resources  to  be 
scheduled  as  well  as  precedence  relationships 
between  tasks  and  training  units.  These  features 
of  the  problem  occasionally  required  two  or 
more  battalions  to  work  together  and  share 
resources  in  accomplishing  some  training  tasks. 
The  authors  developed  a  two-phase  heuristic 
program  for  scheduling  training  activities  and 
resources. 

The  structure  of  the  basic  training  schedul¬ 
ing  problem  (see  Section  2)  is  slightly  similar  to 
the  economic  lot-sizing  problem  (ELSP)  from 
manufacturing  first  described  by  Wagner  and 
Whitin  (1958)  and  Manne  (1958).  Notable  solu¬ 
tion  methods  and  extensions  to  the  problem  dis¬ 
cussed  by  Graves  (1981),  Blackburn  and  Millen 
(1982),  Afentakis,  Gavish,  and  Karmarkar  (1984), 
Billington,  McClain,  and  Thomas  (1986),  among 
others,  suggested  ideas  for  modeling  and  solv¬ 
ing  the  basic  training  problem.  In  another  related 
area,  Rao  (1990)  developed  a  planning  model 
that  minimized  fixed  and  variable  recruitment 
and  training  costs  for  a  personnel  management 
system.  Rao's  approach  motivated  our  develop¬ 
ment  of  a  cost  functional  for  the  basic  training 
scheduling  problem. 

2.  MODEL  FORMULATION 

This  section  presents  the  mathematical 
model  of  the  basic  training  problem.  Practical 
aspects  of  Basic  Combat  Training  essential  to 
proper  development  of  a  dynamic  training 
system  model  are  discussed  first. 

Estimating  the  Weekly  Arrival  of 
New  Trainees 

For  the  version  of  the  basic  training  problem 
presented  here,  we  estimate  trainee  arrivals 
ahead  of  time  by  week  t  and  year  j  for  the  entire 
planning  horizon  of  Tj  weeks.  Recruiting  contin¬ 
ues  throughout  the  year  and  focuses  primarily 
on  young  people  in  their  final  year  of  high 
school.  Force  structure  personnel  requirements 
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Figure  3:  Percent  of  annual  recruit  arrivals  by  month. 


may  cause  annual  recruiting  targets  to  vary  from 
year-to-year.  However,  analysis  of  historical 
recruitment  data  reveals  that  the  distribution  of 
weekly  trainee  arrivals  at  all  training  installa¬ 
tions  throughout  the  year  remains  relatively  sta¬ 
tionary  (see  Figure  3).  Weekly  trainee  arrivals  are 
forecasted  by  applying  the  historical  distribution 
of  trainee  arrivals  to  the  annual  recruiting  targets 
for  each  year  j  of  the  planning  horizon.  The 
absence  of  a  random  disturbance  makes  this 
formulation  of  the  basic  training  problem 
completely  deterministic. 

Dynamics  of  Varying  Training 
Company  Strength 

Company  strength,  the  number  of  trainees 
assigned  per  training  company,  is  bounded 
below  at  150  and  above  at  250  trainees.  In  prac¬ 
tice,  training  managers  assign  all  companies 
scheduled  to  start  training  in  the  same  week 
equal  strengths.  This  rule  simplifies  logistical 
problems  relating  to  training  program  manage¬ 
ment  and  has  been  incorporated  into  the  basic 
training  model  presented  here. 

Determining  the  company  strength  for  a 
given  week  requires  two  critical  elements  of 
information.  These  are  the  number  of  trainees 
who  report  for  training  in  week  t  and  the  num¬ 
ber  of  training  companies  available  at  the  begin¬ 
ning  of  week  t  to  start  training  new  soldiers. 
However,  the  number  of  training  companies 
available  to  start  training  in  week  t  depends 
upon  past  company  strength  decisions.  In  any 
week,  it  is  possible  for  company  strength  deci¬ 
sions  from  previous  weeks  to  cause  a  training 
company  shortfall.  A  shortfall  occurs  when  the 


number  of  training  companies  is  not  sufficient  to 
handle  the  arrival  of  trainees,  also  referred  to  as 
training  load. 

Instructor-to-Trainee  Ratio 

Basic  training  is  designed  to  be  highly  stress¬ 
ful  for  new  trainees.  Training  managers  rely  on 
high  quality  instruction  and  close  supervision  of 
trainees  to  somewhat  offset  the  negative  effects 
of  stress  on  learning.  In  practice,  training  man¬ 
agers  attempt  to  keep  the  ratio  of  instructors-to- 
trainees  around  l-to-16  or  higher,  if  possible.  This 
results  in  company  strengths  of  approximately 
200  trainees  per  company.  This  ratio  is  the  pri¬ 
mary  performance  measure  for  the  optimal  and 
automated  heuristic  scheduling  methods 
presented  here. 

Compressing’the-Load 

Each  week  as  trainees  report  to  basic  train¬ 
ing  installations,  they  are  assigned  to  training 
companies.  This  fill  week  runs  from  Saturday 
through  midnight  Thursday.  Basic  Combat 
Training  begins  on  Friday  and  normally  lasts  8 
weeks.  At  the  completion  of  basic  training,  train¬ 
ing  managers  schedule  a  maintenance  week  for 
each  training  company  before  starting  the  next 
training  cycle.  This  10- week  sequence  is  called  a 
normal  training  cycle. 

In  some  cases,  it  may  not  be  possible  to  elim¬ 
inate  a  training  company  shortfall  by  adjusting 
company  strengths  alone.  Another  way  to  cor¬ 
rect  the  shortfall  is  to  shorten  the  training  cycle 
length  for  companies  that  started  either  8  or  9 
weeks  earlier.  This  is  done  by  eliminating  either 
the  fill  week  or  the  maintenance  week,  or  both. 
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This  practice,  called  compressing-the-load,  short¬ 
ens  or  eliminates  the  break  between  training 
cycles  and  is  known  to  degrade  training  cadre 
effectiveness.  Therefore,  training  managers  only 
compress  the  load  when  the  demand  for  training 
companies  cannot  be  met  by  adjusting  company 
strengths. 

Mathematical  Notation 

j:  year  of  the  planning  horizon,;  e  {1,2,. 
t:  week  of  a  given  year  ;,  t  e  (1,2,..., T^) 
where  Ty  is  the  number  of  weeks  in  year;. 
8{t):  trainee  show  rate  for  week  t  where  0  ^ 
d(t)  <  1. 

pit):  relative  frequency  distribution  of  trainee 
arrivals  over  week  t  of  any  year. 

Dji  number  of  training  companies  deactivat¬ 
ed  in  year ;. 

Mji  number  of  training  companies  available 
at  the  beginning  of  year;. 

Rji  recruiting  objective  for  year ;  determined 
by  Department  of  the  Army. 
ditj):  number  of  training  companies  to  deacti¬ 
vate  in  week  t  of  year ; . 

r(fy):  estimated  number  of  trainees  that  show 
up  for  training  in  week  t  of  year;. 
x(tj):  strength  of  companies  starting  in  week  t 
of  year;. 

X:  upper  bound  for  training  company 
strength. 

x:  lower  bound  for  training  company 
strength. 

yitj):  training  cycle  length  for  companies  start¬ 
ing  in  week  t  of  year ;. 

Y:  upper  bound  for  training  cycle  length. 
y:  lower  bound  for  training  cycle  length. 
litj):  number  of  idle  training  companies  at  the 
beginning  of  week  t  of  year ;. 

I:  upper  bound  for  the  idle  training 
company  constraint. 

Modeling  Constraints 

X  <  x(tj  )<X:  company  strength  constraint.  (1) 
y  <  yijtj )  <  7  :  training  cycle  constraint.  (2) 

Mj  >  Dj :  deactivation  scenario  constraint.  (3) 
d{tj)>  0  V  {tj) :  company  deactivation  constraint. 

(4) 

0  <  I{tj)  <  /  V  (t,j) :  problem  feasibility  constraint. 

(5) 


r{tj)  =  5(/) pit)  Rj :  expected  number  of  (6) 

trainees  to  arrive  for  training  in  week  t  of 
year  ;.  The  recruiting  objective  Xy,  deter¬ 
mined  by  the  Department  of  the  Army,  is 
greater  than  the  number  of  new  soldiers 
required  to  meet  the  personnel  needs  of  the 
Army  in  a  given  year;. 

3.  OPTIMAL  DECISIONS  USING 
DYNAMIC  PRDGRAMMING 

We  simplified  the  dynamic  programming 
(DP)  formulation  of  the  basic  training  problem 
by  removing  the  training  company  deactivation 
decision  d(fy).  This  was  done  by  requiring  these 
decisions  to  be  made  prior  to  implementation  of 
the  dynamic  programming  algorithm. 

Stages 

In  the  basic  training  problem,  stages  are 
denoted  by  week  t  and  year  ;.  The  planning  hori¬ 
zon  consists  of  Tj  discrete,  identical  weeks  where 
f  e{l,2,...,Ty}  and;  e {1,2,...,/}. 

Scheduling  Decisions  and  Scheduling 
Policy 

In  any  week  t  e  {1,  2,...,Ty  -  Ij,  company 
strength  xit)  and  training  cycle  length  yitj)  deci¬ 
sions  for  all  training  companies  are  made  at  the 
beginning  of  the  week.  We  require  that  xit^)  e  fl 
and  yitj)  e  A.  H  and  A  are  decision  spaces  con¬ 
sisting  of  bounded  integer  sets  specified  by  the 
company  strength  constraint  x  <  x(fy)  <  X  and 
the  training  cycle  constraint  y  <  i/(fy)  ^  Y,  respec¬ 
tively.  The  subsets  of  feasible  decisions  to  take  at 

each  stage  t  are  denoted  by  xh’  and 

y[0’  This  notation  indicates  that  deci¬ 

sion  elements  belonging  to  these  two  subspaces 
depend  upon  both  the  stage  t  and  the  state  /(f  ) 
of  the  basic  training  system.  A  sequence  of  such 
decisions,  denoted  by  tt,  is  represented  by 
x(\,l  x(2i),...AT/-1);  1 

^(10,7(2i),.*.,7(r;-l)  J 

The  set  of  all  feasible  sequences  IT  consists  of  all 
solutions  satisfying  constraints  (1)  through  (5) 
above. 
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Objective  Function 

Here  we  are  interested  in  obtaining  the  opti¬ 
mal  training  schedule  that  maximizes  the  "qual¬ 
ity"  of  training  as  measured  by  the  ratio  of 
instructors  to  trainees.  In  practice,  we  maximize 
the  instructor-to-trainee  ratio  by  minimizing 
company  strengths.  Coincidentally  this  simulta¬ 
neously  minimizes  idle  training  companies 
which  serves  as  a  measure  of  training  resource 
utilization. 

Assuming  one  instructor  per  training  com¬ 
pany  for  simplicity  then  for  each  sequence  of 
decisions  77  e  11  there  is  a  corresponding  value 
Ktt  based  on  the  instructor-to-trainee  ratio  that 
provides  a  measure  of  quality  to  be  maximized. 
This  measure  is  given  by 


The  optimal  sequence  of  decisions  it*  is  the  one 
that  maximizes  the  following  objective  function 
for  a  fixed  initial  state: 


(8) 


state  Transition  Equation 


The  state  of  the  basic  training  system 
evolves  according  to  the  following  balance  equa¬ 
tion  for  idle  training  companies: 


I(‘i  + 1)  =  /fy,  I(<j%  4?y)]=  Ij(t) 


rjtj)  y  r(tj-I) 


(9) 


/['■y.  Htj),  Jc(?y)]  is  explicitly  defined  as  an  equiv¬ 
alent  representation  of  the  right  hand  side  of  (9). 

rjtj) 

^(0  )  is  an  estimate  of  the  number  of  compa¬ 
nies  to  begin  training  in  week  t  of  year  /. 
represents  the  number  of  companies  that  become 
available  at  the  beginning  of  week  f+1  to  start 
another  training  cycle  having  just  completed  one 
that  began  either  8,  9,  or  10  weeks  earlier  (see 
Compressing-the-Load).  The  possible  values  for  I  e 
L  are  contained  within  the  set  L  e  {(10),  (10,9), 
(10,9,8)}  (see  McGinnis  (1994),  Section  2.3,  for 
details). 

To  maintain  the  integer  value  of  J(fy  +  1)  in 
(9),  we  round  as  follows: 


For  x{tj)<x:  + 


r(tj) 

+ 

;  (10) 


For 


+ 

x(tj) 

1  e  L  ^^9  0 

.(11) 


When  the  current  company  strength  constraint  is 
tight,  that  is,  an  equality  constraint  exists  at  the 
upper  bound,  then  the  fractional  part  of  a  train¬ 
ing  company  is  rounded  up  as  denoted  by  the 
ceiling  operator  [*].  When  all  training  companies 
are  at  full  strength,  the  number  of  trainees  repre¬ 
sented  by  the  fractional  part  of  a  training  compa¬ 
ny  can  only  begin  basic  training  if  an  additional 
training  company  is  scheduled  to  start.  In  all 
other  cases,  the  fractional  part  may  be  dropped 
as  denoted  by  the  floor  operator  [*].  This  rule  is 
consistent  with  how  training  managers  size 
training  companies  in  practice  when  all  training 
companies  have  not  been  filled  to  capacity  For 
simplicity,  the  floor  and  ceiling  operators  of  (10) 
and  (11),  respectively,  will  not  be  repeated  for 
every  reference  to  training  company  computa¬ 
tions.  However,  it  is  to  be  understood  that  these 
rules  are  in  effect  throughout  the  paper. 

state  Augmentation 


Company  strength  xitj)  and  cycle  length  y(tj) 
decisions  depend  upon  past  information  that 
cannot  be  summarized  in  l{tj  +  1)  alone.  State 
augmentation  makes  additional  information 
needed  for  making  scheduling  decisions  avail¬ 
able  in  the  current  week  (see  Bertsekas).  This 
requires  incorporating  additional  variables  into 
the  formulation  of  the  problem  that  can  signifi¬ 
cantly  increase  the  number  of  computations 
required  to  generate  an  optimal  solution  as  well 
as  the  amount  of  computer  memory  required. 
Therefore,  the  state  space  is  augmented  by  only 
the  minimum  number  of  variables  necessary  to 
make  a  decision  in  each  week.  For  example,  con¬ 
sider  a  simplified  version  of  the  problem  where 
training  cycle  length  is  fixed  at  y(ty)  —  10.  This 
eliminates  training  cycle  length  as  a  decision  ele¬ 
ment  but  it  also  creates  a  9-week  time  lag  in 
information  about  previous  x{tj)  decisions  need¬ 
ed  to  make  company  strength  decisions  in  the 
current  week.  The  minimally  augmented  state  of 
the  system  for  this  case  is  given  by  (12). 
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x{tj)  s\lj) 

S^(lj+l) 

t-- 

1 

11 

O! 

s^Oj+l) 

1 

II 

= 

s^(/j)  =  x(tj-5) 

s^itj+l) 

=  ^(fj-4) 

s^Oj+l) 

=  x(tj-2) 

s\tj+\) 

1 

II 

oo 

1 - 

+ 

1  _ 

x{tj) 

One  can  think  of  (^■)  j  as 

"registers"  for  temporarily  storing  the  required 
information  as  the  system  evolves  (see  Bert- 

sekas).  The  sequence  (/y)  j 

constitutes  the  state  of  the  system  in  our 
formulation. 

Estimating  the  Size  of  the  Problem 

A  combinatorial  explosion  of  the  state  space 
occurs  when  attempting  to  obtain  an  optimal 
solution  to  the  real-world  basic  training  problem 
using  exact  methods  such  as  complete  enumera¬ 
tion  or  dynamic  programming.  The  augmented 
state  space  for  a  single  week  of  the  basic  training 
problem,  based  on  1990  training  data,  is  estimat¬ 
ed  by  enumeration  of  the  following  state 
variables. 


•  Maximum  number  of  idle  training 

companies /(ty  +  1): 

•  Total  company  strength  values 

130. 

x(tj  -  1),. .  .,x{t-  -  9)  for  the 
augmented  state: 

(101)^. 

•  Maximum  number  of  training  cycle 

lengths  y(tj): 

3. 

This  generates  an  upper  bound  estimate  for 
each  week  t  of  the  state  space  of  approximately 
4.27x10^^  possible  states  (i.e.,  3xl30x(101)^). 
Using  complete  enumeration  to  solve  a  two-year 
scheduling  problem  increases  the  size  of  the 
problem  by  approximately  two  orders  of  magni¬ 
tude.  On  the  other  hand,  dynamic  programming 
may  substantially  reduce  the  amount  of 


enumeration  required  to  obtain  an  optimal  solu¬ 
tion.  It  does  this  by  avoiding  decision  sequences 
that  cannot  possibly  be  optimal  and  by  solving 
the  problem  one  stage  at  a  time.  However,  as 
Table  3  illustrates,  the  potential  size  of  the  aug¬ 
mented  state  space  for  the  real-world  problem, 
or  a  reduced  problem,  remains  quite  large.  Other 
exact  methods  such  as  integer  and  mixed  integer 
programming  suffer  from  similar  problems  (see 
McGinnis,  1994).  These  obstacles  motivated  the 
development  of  efficient  heuristics  for  solving 
the  basic  training  scheduling  problem. 

4.  HEURISTIC  APPROACHES 

The  heuristic  procedure  presented  here  is 
applied  in  three  phases.  Phase  I  assumes  an  ini¬ 
tial  training  requirement  for  each  week  t  denoted 
by  {r(l^),  r(2^),...,r(Tj)}.  This  is  estimated  by 
applying  the  historical  distribution  of  trainee 
arrivals  from  Figure  3  to  the  recruiting  target  Rj 
for  each  year  according  to  (6).  Next,  company 
strengths  and  training  cycle  lengths  are  initial¬ 
ized  at  X  and  i/,  respectively  Then  training 
companies  are  scheduled  in  each  week  to  meet 
the  training  requirement.  Idle  training  compa¬ 
nies  I(tj)  are  also  computed  for  each  week.  If 
J(fy)  <  0  in  any  week  f,  then  the  current  scenario 
is  infeasible.  Otherwise  ^(0  )-^  ^’7  and  a  feasi¬ 

ble  schedule  must  exist  for  the  resources  current¬ 
ly  available. 

Next,  company  strengths  and  training  cycle 
lengths  are  reinitialized  at  x  and  Y,  respectively. 
This  may  cause  a  training  company  shortfall  in 
some  weeks  where  I(tj)  <  0.  At  this  point  of 
Phase  I  a  single-pass  heuristic  (SPH)  makes  one 
forward  pass  through  the  planning  horizon  to 
correct  training  company  shortfalls  in  each  week 
where  I{tj)  <  0.  A  policy  iteration  algorithm  is 
applied  a  finite  number  of  times  to  x(tj)  and  y(fy) 
until  I{tj)  >  0.  The  single-pass  heuristic  repeats 
the  policy  iteration  algorithm  in  each  week  of  the 
planning  horizon  where  I(tj)  <  0  thereby  gener¬ 
ating  an  initial  feasible  training  schedule.  The 
feasible  scheduling  policy  for  Phase  I,  signified 
by  superscript  1,  is  specified  by 

y'(hly\20 . y'(Tj-i)  j 

Phase  II  considers  options  for  changing  the 
level  of  resources  available  to  train  soldiers.  The 
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version  of  the  model  presented  here  only  consid¬ 
ers  decisions  that  change  the  level  of  training 
companies  available.  However,  the  model  is  eas¬ 
ily  modified  to  consider  other  reusable 
resources.  In  practice,  if  the  training  scenario 
does  not  involve  resource  level  changes,  then 
Phase  II  may  be  omitted.  The  resource 
scheduling  policy  for  Phase  II  is 


K 


2 


dHtj)  represents  the  company  deactivation 
decision  in  week  t  and  superscript  2  denotes 
Phase  II. 

Experiments  with  the  heuristic  procedure 
have  shown  it  is  possible  to  improve  resource 
schedules  from  Phase  I.  This  observation  led  to 
the  development  of  a  multi-pass  heuristic  (MPH) 
that  improves  the  resource  scheduling  policies 
obtained  from  the  single-pass  heuristic  through  a 
modified  policy  improvement  step  that  further 
decreases  company  strengths. 

Phase  III  uses  the  initial  feasible  schedule 
from  Phase  II  as  its  starting  point  or  from 
Phase  I  if  Phase  II  is  omitted.  The  multi-pass 
heuristic  iteratively  revises  the  company 
strength  policy,  week-by-week,  working  sequen¬ 
tially  backward  through  the  planning  horizon. 
Within  a  week  the  modified  policy  improvement 
algorithm  decreases  company  strength  x{tj)  by 
one  step  of  size  n  at  a  time  until  no  further 
improvement  to  the  objective  function  is  possi¬ 
ble.  The  algorithm  stops  when  either  company 
strength  reaches  its  lower  bound  x  or  Ktj)  =  0. 
The  resource  scheduling  policy  obtained  at  the 
completion  of  Phase  III  is 


x\2,),...,x\Tj-\); 

d\hyd\20,...,d^(Tj-l) 


x^(f^)  denotes  the  "best"  suboptimal  company 
strength  decision  in  week  t  given  all  previous 
decisions.  yHtj)  and  (fi(tj)  are  the  training  cycle 
and  training  company  deactivation  decisions 
from  Phase  II. 


5.  DECISION  SUPPORT  SYSTEM 
(DSS) 

The  logistical  complexities  of  the  Army's  ini¬ 
tial  entry  training  program  create  numerous 
practical  decision  problems.  In  many  of  these 
decision  situations  the  best  solutions  are  not 
obvious.  For  example,  decision  makers  may  be 
confronted  by  multiple  competing  objectives.  In 
other  situations  decisions  may  depend  upon  a 
sequential  decision  process  complicated  by 
precedence  constraints,  or  decisions  may  need  to 
be  evaluated  over  different  planning  horizons. 
The  Decision  Support  System  for  Army  Basic 
Combat  Training  Resource  Management  per  Year,  or 
ARMY,  has  been  developed  to  study  and 
methodically  solve  a  broad  range  of  basic 
training  scheduling  problems. 

At  the  highest  level  of  the  Army, 
Department  of  the  Army,  force  planners  are 
responsible  for  properly  manning  the  force. 
Force  planners  forecast  recruitment  objectives 
based  upon  personnel  requirements  derived 
from  future  military  force  structure.  The  deter¬ 
mination  of  recruitment  objectives  generates 
annual  recruiting  targets  that,  in  turn,  drive 
training  resource  requirements.  Recent  events 
such  as  force  structure  downsizing  and 
realignment  and  closure  of  military  bases  have 
complicated  the  force  planning  process.  The 
ARMY  system  has  been  designed  to  analyze  the 
impact  of  these  events  in  terms  of  recruitment 
objectives  and  training  program  throughput.  For 
example,  demand  for  training  company 
resources  is  determined  by  recruiting  objectives 
that  represents  force  structure  personnel  require¬ 
ments.  The  ARMY  system  can  also  forecast  train¬ 
ing  program  throughput  based  on  the  level  of 
training  resources  available  to  conduct  training 
that,  in  part,  reflects  training  base  realignment 
and  closure  decisions.  Both  recruiting  objectives 
and  training  companies  are  model  parameters 
that  are  easily  changed  by  the  system  user. 

At  TRADOC  Headquarters,  training  pro¬ 
gram  managers  acquire  and  distribute  training 
resources  needed  by  training  installations  to 
accomplish  Army  training  missions.  The  deci¬ 
sion  support  system  can  help  TRADOC  training 
managers  determine  training  loads  for  each 
training  installation.  The  system  estimates  train- 
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ing  costs  for  justifying  training  resource  require¬ 
ments  and  costs  to  Department  of  the  Army.  The 
system  can  also  support  studies  and  planning  for 
special  contingencies  such  as  force  mobilization 
and  future  changes  to  training  programs  due  to 
force  structure  downsizing  and  base  closures. 

At  the  installation  level,  the  decision  support 
system  can  help  training  program  managers 
resolve  resource  scheduling  problems  that  affect 
training  program  execution.  The  ARMY  system 
can  develop  precise  training  resource  schedules 
that  meet  Department  of  the  Army  training 
requirements  or  determine  that  a  feasible  sched¬ 
ule  does  not  exist  for  a  given  a  training  scenario. 
The  DSS  can  also  help  installation  training  man¬ 
agers  identify  and  resolve  training  resource 
shortfalls.  Finally,  the  system  enables  training 
managers  to  generate  efficient  training  resource 
schedules  where  training  resources  are  highly 
utilized. 

A  future  area  of  change  for  the  basic  training 
program  will  be  the  integration  of  computer  and 
information  technologies  into  the  training  envi¬ 
ronment.  For  example,  the  Army  is  presently 
developing  a  family  of  high  technology,  comput¬ 
er-based  training  facilities  called  the  Combined 
Arms  Tactical  Trainer  (see  McGinnis  and  Phelan, 
1996).  These  facilities  will  enable  army  units  at 
the  same  or  different  locations  to  conduct  realis¬ 
tic,  interactive  training  in  a  virtual  environment 
featuring  a  digitized  battlefield.  These  systems 
are  likely  to  be  used  in  initial  entry  training  as 
well.  The  Army  is  researching  and  developing 
new,  innovative  methods  for  teaching  new  sol¬ 
diers  the  technical  skills  needed  for  operating 
and  maintaining  high-technology  systems  of  the 
future.  The  curricula  for  future  initial  entry  train¬ 
ing  programs  will  likely  be  substantially  differ¬ 
ent  from  current  curricula.  In  this  area,  the 
ARMY  system  may  be  used  to  study  potential 
training  program  changes  such  as  varying 
course  lengths,  classroom  sizes,  training  compa¬ 
ny  strengths,  training  program  throughput, 
training  resource  requirements,  and  training 
program  costs. 

5.1  Decision  Support  System 
Development 

Decision  support  system  development  was 
accomplished  through  four  sequential,  over¬ 
lapping  tasks. 


1.  Functional  description  of  the  system. 

2.  Preliminary  design  of  system  architecture 
and  system  modules. 

3.  Development  of  a  system  prototype. 

4.  Full  system  development. 

In  Task  1,  the  primary  functions  of  the  deci¬ 
sion  support  system  for  supporting  the  schedul¬ 
ing  decision  process  were  identified.  In  Task  2, 
the  system  architecture  was  represented  graphi¬ 
cally  through  a  set  of  interconnected  modules. 
The  modules  embody  the  functional  require¬ 
ments  of  the  system  identified  in  Task  1.  Four 
major  issues  were  addressed  during  system 
architecture  design. 

1.  System  and  module  functionality. 

2.  System  and  module  data  exchange 
requirements. 

3.  Module  procedures,  logic,  and  rules  for 
performing  scheduling  operations, 

4.  Data  generation,  storage,  and  retrieval 
requirements. 

Figure  4  illustrates  the  DSS  architecture  and 
system  modules. 

In  Task  3  module  prototypes  were  devel¬ 
oped  and  implemented  in  a  computer  spread¬ 
sheet  called  LOTUS  1-2-3  for  Windows.  The 
spreadsheet  environment  was  ideal  for  stream¬ 
lining  the  thousands  of  repetitive  calculations 
required  to  generate  a  weekly  training  resource 
schedule.  Scheduling  procedures  and  rules  for 
controlling  the  flow  of  data  between  modules 
and  policy  improvement  routines  were  pro¬ 
grammed  using  advanced  macros.  The  spread¬ 
sheet  software  also  provided  built-in  tools  for 
statistical  analysis  of  scheduling  output.  Once 
the  system  modules  were  performing  as  expect¬ 
ed,  information  links  were  established  between 
the  modules  for  dynamic  data  exchange.  Full 
system  testing  and  system  documentation  in 
Task  4  concluded  DSS  development. 

5.2  Description  of  the  Decision 
Support  System  Modules 

As  shown  in  Figure  4,  the  centerpiece  of  the 
ARMY  system  is  the  dynamic  mathematical 
model  of  the  basic  training  program.  The 
descriptive  module  names  indicate  the  primary 
functionality  of  each  module. 
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Figure  4:  Decision  support  system  architecture  and  system  modules. 


The  Resource  Scheduler  Module  first  forecasts 
the  number  of  trainees  to  arrive  for  training  each 
week  and  then  computes  the  number  of  training 
companies  required  to  start  training.  Next,  the 
module  schedules  basic  training  companies,  or 
other  resources,  in  each  week  of  the  planning 
horizon.  Scheduling  rules,  constraints,  and  user- 
specified  scheduling  conditions  describing  the 
initial  state  of  the  training  system  are  used 
within  this  module  to  schedule  resources. 

The  Policy  Improvement  Module  invokes  the 
fully  automated  single-pass  and  multi-pass 
heuristic  procedures  to  generate  a  good,  feasible 
training  resource  schedule,  if  one  exists.  This 
eliminates  the  need  for  a  system  user  to  interact 
with  the  computer  spreadsheet  model  as  was  the 
case  when  using  the  partially  automated  the 
manual  heuristics-used-in-practice  developed  pre¬ 
viously  by  McGinnis  (1989). 

The  Numerical  Analysis  Module  analyzes 
scheduling  data,  computes  scheduling  statistics, 
and  summarizes  scheduling  information.  The 
Graphical  Analysis  Module  graphs  scheduling 
information  and  statistics.  The  Report  Generation 
Module  produces  numerical  and  graphical  sched¬ 
uling  output  tailored  to  the  decision  making 
needs  of  training  managers.  Section  6  gives 
examples  of  numerical  and  graphical  output 


from  the  system.  Output  from  an  illustrative 
scheduling  session  with  the  ARMY  system  is 
given  in  Appendix  C  of  McGinnis  (1994). 

The  Resource  Costing  Module  estimates  vari¬ 
ous  training  program  costs  from  the  training 
resource  schedules  generated  by  the  decision 
support  scheduling  system.  Cost  measures  cur¬ 
rently  determined  by  the  system  include  the 
following. 

•  Total  annual  training  program  costs.  Total  pro- 
gram  costs  are  estimated  for  each  year  of  a 
two-year  scheduling  horizon.  Annual  cost  cat¬ 
egories  include  fixed  and  variable  costs  by 
training  resource. 

•  Annual  training  program  cost  variance.  The 
annual  training  program  cost  variance  repre¬ 
sents  cost  differences  between  the  two  years  of 
scheduling  data  generated  by  ARMY.  The  sys¬ 
tem  expresses  cost  variances  in  constant  dol¬ 
lars  and  also  as  percent  differences  in  training 
costs  from  year-to-year.  These  differences 
reflect  changes  to  training  installations  due  to 
base  closure  and  realignment  or  force  struc¬ 
ture  changes  measured  as  explained  above. 
Comparisons  of  annual  cost  variances  are 
made  by  total  program  cost,  fixed  and 
variable  costs,  and  training  resource  costs. 
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1 .  In-processing  activities 

6.  Laundry  services 

2.  Basic  training  support 

7.  Food  services 

3.  Supply  operations 

8.  Personnel  support 

4.  Maintenance  of  basic  training  equipment 

9.  Ammunition  for  weapons  qualification 

5.  Transportation  services 

10.  Utilities 

Table  1.  Cost  items  measured  using  the  ARMY  system. 


•  Average  training  program  costs.  The  system 
computes  various  average  cost  estimates  per 
training  cycle.  These  include  an  average  vari¬ 
able  training  cost  for  the  entire  training  pro¬ 
gram  as  well  as  by  training  battalion, 
company,  and  trainee.  The  system  also  esti¬ 
mates  average  variable  costs  per  training  cycle 
by  training  company  and  resource.  Finally,  an 
average  program  cost  per  trainee  and  an  aver¬ 
age  program  cost  per  trainee  by  training 
resource  are  also  estimated. 

Cost  data  for  this  study  was  provided  by  the 
Directorate  of  Resource  Management^  (DRM)  of 
Fort  Benning,  Georgia,  based  on  a  1993  study  of 
basic  training  resource  utilization  conducted  at 
Fort  Benning  by  the  DRM.  The  1993  Fort  Benning 
Study  measured  14  training  resource  costs  for  a 
single  basic  training  battalion.  The  training  bat¬ 
talion  consisted  of  a  headquarters  company  and 
five  basic  training  companies.  Cost  data  were 
collected  during  six  consecutive  training  cycles. 
Ten  cost  items  from  the  Fort  Benning  study  were 
used  in  this  study  to  illustrate  the  costing 
methodology.  The  cost  items  are  given  in  Table  1. 

6.  RESULTS 

Section  6.1  compares  scheduling  results 
from  the  heuristic  methods  for  12  realistic  test 
scenarios.  The  number  of  training  companies 
available  in  each  scenario  reflect  potential  base 
realignment  and  closure  decisions  that  differenti¬ 
ate  the  test  scenarios.  The  results  are  compared 
using  three  performance  measures:  processing 
time  to  generate  a  training  schedule,  training 
quality,  and  training  resource  utilization.  See 
McGinnis  (1994)  for  a  comparison  of  scheduling 
results  based  on  basic  training  costs.  Three  sets 
of  scheduling  results  are  presented  for  each  test 
scenario  and  performance  measure.  One  set  of 
results  was  generated  using  a  semi-automated 
Heuristics-Used-In-Practice  (HUIP)  procedure 
developed  previously  by  McGinnis  (1989).  The 


other  two  sets  came  from  the  Single-Pass 
Heuristic  (SPH)  alone  and  the  combined  Single- 
Pass  and  Multi-Pass  Heuristic^  (SPH-MPH). 

Section  6.2  compares  optimal  scheduling 
solutions  from  dynamic  programming  with 
results  from  the  combined  Single-Pass  and 
Multi-Pass  Heuristic  procedure.  As  discussed  in 
Section  3,  the  size  of  the  basic  training  schedul¬ 
ing  problem  precluded  implementation  of  exact 
solution  methods  for  the  real-world  problem. 
Nevertheless,  we  implemented  dynamic  pro¬ 
gramming  for  a  much  simplified  version  of  the 
basic  training  problem.  These  results  serve  as  a 
yardstick  for  measuring  the  quality  of  feasible 
schedules  obtained  using  the  heuristic. 

Test  Scenarios 

The  test  scenarios  for  this  study  reflect  1990 
training  base  structure  and  approximate  recruit¬ 
ing  targets  for  1989  and  1990.  Table  2  gives  the 
number  of  training  companies  deactivated  by 
year  and  scenario  (see  and  D^^^q).  The 

annual  recruiting  targets  for  the  scheduling  hori¬ 
zon  were  ^1959  =  136,000  and  —  128,000. 
The  scheduling  system  was  initialized  with  130 
training  companies  at  the  beginning  of  the  first 
year  of  the  planning  horizon  and  10- week  train¬ 
ing  cycle  lengths  throughout.  Training  company 
strengths  for  all  test  scenarios  were  initialized  at 
200  trainees  per  company  for  the  HUIP  proce¬ 
dure  and  150  trainees  for  the  SPH  and  SPH- 
MPH  procedures  except  for  Scenarios  10  and  12. 
In  these  two  cases,  the  company  strengths  of  all 
three  scheduling  methods  were  initialized  at  150 
and  200  trainees  per  company,  respectively. 

6.1  Heuristic  Scheduling  Results 

This  section  evaluates  training  resource 
schedules  from  the  heuristic  scheduling 
procedures  using  three  performance  measures. 
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Table  2.  Numerical  scheduling  results  for  the  heuristic  methods. 


Schedule  processing  time.  The  time  to 


obtain  a  feasible  schedule. 


2.  Training  quality.  Measured  in  terms  of  an 

Tj  1 

instructor-to-trainee  ratio  X  x 

where  we  assume  one  instructor  per  training 
company  for  simplicity.  The  utopian  value  for 
training  quality  based  on  company  strengths  of 
150  trainees  and  a  two-year  scheduling  horizon 
is  0.64,  so  named  because  it  will  not  usually  be 
obtainable  under  normal  circumstances. 


3.  Training  resource  utilization.  Measured 
by  the  average  number  of  idle  training  compa¬ 
nies  per  week .  1 

T, 

Scheduling  Results 

The  first  author  conducted  all  computational 
tests  on  a  486/66  desktop  computer.  Table  2 
gives  numerical  results  by  training  base  scenario 
{Seen)  and  scheduling  method  (HUIP,  SPH,  SPH- 
MPH)  for  the  performance  measures  cited  above. 
The  test  cases  were  limited  to  scenarios  where 
the  scheduling  methods  generated  feasible 
schedules.  This  established  a  common  basis  for 
comparing  the  scheduling  results  by  heuristic 
method  and  performance  measure.  The  entries 
in  each  cell  of  columns  and  of  Table  2 
indicate  the  number  of  training  companies 
"deactivated"  each  year.  Processing  Time  reflects 


Tj 

1%) 


the  time  in  minutes  and  seconds  to  generate  a 
training  resource  schedule.  Ratio  gives  the 
instructor-to-trainee  ratio  values  summed  over 
the  two-year  planning  horizon  of  Tj  weeks.  Idle 
Co  represents  the  number  of  idle  training  compa¬ 
nies  averaged  over  the  planning  horizon  and 
rounded  to  the  nearest  whole  training  company 

Heuristic  Scheduling  Processing  Time 

Figure  5  graphs  training  schedule  processing 
times  for  the  heuristic  procedures.  For  Scenarios 
1  through  11,  the  SPH  procedure  generated 
schedules  6.4  times  faster  than  the  FlUIP  proce¬ 
dure  and  1.7  times  faster  than  the  combined 
SPH-MPH  procedure.  The  combined  SPH-MPH 
procedure  was  3.7  times  faster  than  the  HUIP 
procedure  on  average.  In  Scenario  12,  company 
strengths  were  initialized  at  200  trainees  model¬ 
ing  the  initial  condition  of  the  heuristics-used-in- 
practice.  Although  this  initial  condition 
dramatically  reduced  schedule  generation  time 
using  the  SPH  procedure,  it  also  led  to  signifi¬ 
cantly  poorer  SPH  results  in  terms  of  training 
quality  and  training  resource  utilization  (see 
Table  2).  It  also  substantially  increased  the  pro¬ 
cessing  times  for  the  combined  SPH-MPH  proce¬ 
dure  since  many  more  iterations  of  the  policy 
improvement  algorithm  were  necessary  to 
improve  the  company  strength  policy 

Instructor-to-Trainee  Ratio 

As  discussed  in  Section  2,  the  instructor-to- 
trainee  ratio  reflects  company  strength  decisions 
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Figure  5:  Heuristic  processing  times. 


in  each  week  and  also  serves  as  a  performance 
measure  of  training  "quality."  Therefore,  the 
problem  is  suboptimized  by  making  company 
strengths  as  small  as  possible.  Figure  6  gives  the 
ratio  values  for  the  12  test  scenarios.  On  average, 
the  quality  of  the  SPH  and  SPH-MPH  schedules 
for  Scenarios  1  through  11  were  18.7%  and  19% 
better  than  the  HUIP  schedules,  respectively. 

Although  the  SPH-MPH  improved  SPH 
solutions  by  only  0.3%,  this  modest  improve¬ 
ment  in  the  "quality"  of  training  may  be  signifi¬ 
cant  in  terms  of  total  annual  training  program 
throughput.  In  Scenario  12,  initializing  company 
strengths  at  200  imposes  a  "penalty"  on  the  qual¬ 
ity  of  SPH  scheduling  solutions.  However, 
Scenario  12  also  illustrates  the  how  the  MPH 
procedure  methodically  tightens  company 
strengths  in  each  week  of  the  planning  horizon 
to  improve  the  quality  measure.  The  MPH  made 
a  substantial  16.8%  improvement  in  the  SPH- 


MPH  objective  function  value:  0.470  for  the  SPH 
versus  0.549  for  the  SPH-MPH. 

Figure  7  compares  the  scheduling  results  as 
a  percentage  of  the  utopian  value  where  100% 
corresponds  to  an  instructor-to-trainee  ratio  of 
0.64. 

Figure  7  also  illustrates  the  effects  of  the 
training  company  initial  condition  on  the  "quali¬ 
ty"  of  training  schedules.  Excluding  Scenarios  10 
and  12,  the  HUIP,  SPH,  and  MPH  methods  gen¬ 
erated  solutions  that  were,  on  average,  approxi¬ 
mately  74%,  87.5%,  and  87.8%  of  the  utopian 
training  "quality"  value. 

Average  Number  of  Idle  Training 
Companies 

The  average  number  of  idle  training  compa¬ 
nies  for  each  training  resource  schedule  may  be 
interpreted  as  a  measure  of  training  company 


Figure  6:  Instructor-to-Trainee  Ratios. 
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Figure  7.  Heuristic  solutions  measured  as  a  percentage  of  the  utopian  value. 


utilization.  For  this  measure,  the  objective  is  to  improvement  algorithm  of  Phase  1  increases  x(tj) 
generate  a  training  schedule  that  "minimizes"  in  weeks  where  <0  by  one  step  of  size  n  at  a 
idle  training  companies.  Coincidentally,  this  is  time.  The  instant  the  constraint  becomes  tight, 
achieved  by  "maximizing"  the  instructor-to-  that  is  litj)  =  0,  the  algorithm  stops.  Therefore, 

trainee  ratio  in  each  week.  As  explained  above,  although  the  MPH  procedure  may  improve  the 

this  is  accomplished  by  making  company  company  strength  policy  in  each  week,  it  cannot 

strengths  as  small  as  possible.  As  shown  in  further  reduce  idle  training  companies. 

Figure  8,  the  average  number  of  idle  training  However,  when  training  company  strengths  are 

companies  is  also  highly  dependent  upon  the  initialized  at  values  greater  than  the  lower 

number  of  training  companies  available  for  bound,  such  as  at  200  trainees  in  Scenario  12, 
training.  Obviously,  when  fewer  companies  are  then  there  may  be  numerous  weeks  where  J(fy)  > 

available  to  train  soldiers,  the  remaining  compa-  0.  Refinement  of  company  strengths  in  these 

nies  are  used  more  often  leading  to  a  higher  weeks  by  the  MPH  procedure  may  reduce  idle 

training  company  utilization  and  fewer  idle  training  companies  as  well.  The  SPH-MPH  pro¬ 
companies  in  each  period.  cedure  generated  schedules  that  were  3.8  times 

Figure  8  also  reveals  that  for  all  scenarios  more  efficient  at  utilizing  resources,  on  average, 
except  12,  idle  training  companies  determined  than  the  HUIP  procedure  (excluding  Scenario 
from  the  SPH  and  SPH-MPH  procedures  are  12). 
exactly  equal.  The  company  strength  policy 


Figure  8.  Idle  training  companies 
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6.2  Comparison  of  Optimal  and 
Heuristic  Scheduling  Results 

As  shown  in  Section  3,  the  size  of  the  real- 
world  basic  training  problem  makes  it  impracti¬ 
cal  to  use  exact  solution  procedures  for  solving 
the  problem.  However,  we  implemented  dynam¬ 
ic  programming  for  a  simplified  version  of  the 
problem  in  order  to  compare  heuristic  and  opti¬ 
mal  scheduling  results.  This  required  the 
following  modifications  to  the  problem. 

•  Training  cycle  length  decisions  were 
removed  by  fixing  cycle  lengths  at  y{t^  =  2.  This 
reduced  the  information  time  lag  from  9  weeks 
to  1  week. 

•  The  allowable  number  of  idle  training 
companies  l{t-)  was  reduced  from  130  to  20 
based  on  experimental  results  from  testing  the 
automated  heuristic. 

•  The  allowable  company  strength  deci¬ 
sions  x(tj)  were  reduced  from  101  reflecting  a 
step  size  of  1  to  21  for  a  step  size  of  5. 

These  modifications  reduced  the  size  of  one 
stage  of  the  problem  from  approximately 
4.27x10^®  possible  states  to  9,261  states  (i.e., 
21x21x21).  The  planning  horizon  was  reduced 
from  96  to  48  weeks.  This  established  a  new 
utopian  value  of  0.32  for  the  training  '"quality" 
performance  measure.  Although  shortening  the 


planning  horizon  does  not  affect  the  size  of  the 
state  space,  it  does  reduce  computer  memory 
required  to  generate  and  store  training  schedules 
as  well  as  processing  time. 

Implementation  of  the  DP  Algorithm 

The  state  transition  equation  for  a  one- week 
time  lag  problem  and  a  one-year  planning  hori¬ 
zon  is  given  by 


/(/+!)  =  m  - 


KO 

x(/) 


+ 


K/-1) 

x(t  - 1) 


Company  strengths  x{t)  remain  restricted  to 
the  allowable  set  of  integer-valued  decisions 
X[h  but  modified  as  explained  above. 

Despite  the  seemingly  trivial  nature  of  this  prob¬ 
lem,  implementation  of  the  dynamic  program¬ 
ming  algorithm  for  the  one-period  time  lag 
problem  represents  an  important  first  step 
towards  solving  larger,  more  complex  problems. 
The  steps  to  implement  the  algorithm  for  larger 
problems  remain  essentially  the  same  for  the 
one-week  time  lag  problem  with  two  obvious 
exceptions.  First,  the  size  of  state  space  for  the 
real-world  problem  increases  exponentially  with 
state  space  augmentation.  Second,  the  complexi¬ 
ty  of  implementing  dynamic  programming  to 
solve  the  real-world  problem  increases  substan¬ 
tially  as  additional  variables  are  incorporated 
into  the  state  transition  equation  to  deal  with 
information  time  lag.  Table  3  gives  state  space 
increases  for  information  time  lags  based  on  a 
company  strength  step  of  five. 


Time  Lag  (/) 

X  (t4) 

l(t) 

x{t) 

State  Space 

1 

21 

21 

21 

9,261 

2 

21^ 

24 

21 

222,264 

3 

iC 

27 

21 

5.2x10® 

4 

21"^ 

30 

21 

1.2x10* 

5 

iC 

33 

21 

2.8x10^ 

6 

21^ 

36 

21 

6.5x10’® 

7 

iC 

39 

21 

1.5xl0’^ 

8 

21« 

42 

21 

3.3xl0’* 

9 

21^ 

45 

21 

7.5x10’'’ 

10 

2iio 

48 

21 

1.7xl0’® 

Table  3.  State  space  sizes  for  each  1-period  increase  in  information  time  lag. 
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Dynamic  Programming  Versus 
Heuristic  Results 

The  allowable  initial  states  for  the  dynamic 
programming  solution  to  the  one-period  time 
lag  problem  generated  a  total  of  441  scheduling 
solutions.  Table  4  compares  dynamic  program¬ 
ming  and  heuristic  scheduling  results  for  12  rep¬ 
resentative  test  scenarios.  The  SPH-MPH  and  DP 
columns  give  the  objective  function  values  for 
these  scheduling  procedures,  respectively.  % 
Solution  expresses  the  heuristic  solutions  as  a 
percentage  of  the  dynamic  programming 
solutions  for  each  test  scenario. 

The  results  illustrate  the  influence  of  initial 
scheduling  conditions  for  company  strengths 
and  idle  training  companies  on  the  objective 
function  values  for  the  heuristic  procedure.  On 
average,  the  SPH-MPH  procedure  generated 
scheduling  solutions  that  were  approximately 


91%  of  the  optimal  solution  for  the  one-period 
lag  problem.  Figure  9  illustrates  how  changes  in 
the  initial  values  of  x{t)  for  each  value  of  Kt)  dra¬ 
matically  change  the  "quality"  of  heuristic 
solutions  relative  to  optimal  solutions. 

7.  CONCLUSIONS 

This  paper  solves  a  complex  scheduling 
problem  of  practical  interest  to  the  United  States 
Army.  Specifically,  the  problem  is  one  of  sched¬ 
uling  reusable  training  resources  for  the  Army's 
Basic  Combat  Training  program  over  a  finite 
planning  horizon.  Notable  dynamic  characteris¬ 
tics  of  the  basic  training  system  model  include 
the  following. 

1.  Seasonal  arrival  of  trainees  that  represent 
demand  for  training  resources. 

2.  Varying  training  company  strength. 

3.  Varying  training  cycle  length. 


Test  Scenario 

i(t) 

x(t-l) 

SPH-MPH 

DP 

%  Solution 

1 

0 

150 

.3047 

.3121 

98% 

2 

0 

200 

.2831 

.3103 

91  % 

3 

0 

250 

.2652 

.3027 

88% 

4 

1 

150 

.3071 

.3121 

98% 

5 

1 

200 

.2981 

.3113 

93  % 

6 

1 

250 

.2721 

.3063 

89% 

7 

2 

150 

.3091 

.3121 

99% 

8 

2 

200 

.2942 

.3119 

94% 

9 

2 

250 

.2773 

.3084 

90% 

10 

3 

150 

.3108 

.3121 

99% 

11 

3 

200 

.2983 

.3121 

96% 

12 

3 

250 

.2831 

.3103 

91  % 

Table  4.  Scheduling  results  for  the  1-period  time  lag  problem. 


Figure  9.  Comparison  of  heuristic  and  optimal  solutions  for  a  1-period  time  lag  problem. 
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In  real-world  basic  training,  trainees  are 
given  a  window  of  several  weeks  to  report  for 
training.  This  makes  trainee  arrivals  a  random 
process  that,  in  turn,  makes  the  demand  for 
training  resources  stochastic.  However,  we  esti¬ 
mate  trainee  arrivals  ahead  of  time  for  each 
week  of  the  planning  horizon  which  makes  our 
problem  deterministic. 

7.1  Benefits  of  the  Decision  Support 
System 

The  decision  support  system  permits  system 
users  at  different  levels  of  decision  making  to 
study  a  broad  range  of  practical  problems.  For 
example,  at  Department  of  the  Army,  the  system 
can  be  used  to  examine  issues  such  as  the  impact 
of  consolidating,  closing,  or  reducing  the  size  of 
training  installations  on  training  program 
throughput.  Alternatively,  the  system  can  esti¬ 
mate  training  program  throughput  for  meeting 
future  Army  force  structure  requirements  in 
terms  of  recruitment  objectives.  It  can  also  be 
used  to  evaluate  the  feasibility  of  scenarios  for 
expanding  the  training  base  structure  during 
mobilization.  The  ARMY  system  generates 
potentially  useful  output  such  as  the  average 
number  of  idle  training  companies  that  helps 
determine  whether  the  training  base  is  either 
over  or  under  structured  relative  to  projected 
recruitment  targets.  Finally,  the  system  can  sup¬ 
port  analysis  of  alternatives  for  consolidating 
military  training  programs  into  "joint"  training 
centers  where  military  personnel  from  all 
services  may  be  trained  at  the  same  installation. 

At  Training  and  Doctrine  Command 
(TRADOC)  Headquarters  the  system  can  help 
training  program  managers  evaluate  the  eco¬ 
nomic  impact  of  different  resource  utilization 
policies  if  measured  in  terms  of  training  capacity 
and  training  program  throughput.  This  is  done 
by  varying  system  model  parameters  such  as 
recruiting  levels,  training  program  duration 
(course  lengths),  or  the  level  of  training 
resources  available. 

Finally,  at  the  training  installation  level, 
ARMY  can  generate  precise  training  resource 
schedules,  forecast  training  resource  require¬ 
ments,  and  estimate  training  program  costs. 
ARMY  provides  installation  training  program 
managers  with  a  fully  automated,  computer- 
based  scheduling  system  for  generating  "good" 


weekly  schedules  of  reusable  training  resources 
in  reasonable  time.  These  schedules  have  high 
practical  value  as  a  preliminary  step  to  develop¬ 
ing  executable  training  resource  schedules  for 
day-to-day  operations. 

The  system  employs  practical  scheduling 
performance  measures  to  provide  decision  mak¬ 
ers  with  a  rational  basis  for  selecting  the  '1^est" 
alternative  from  competing,  feasible  ones.  The 
performance  measures  include  training  program 
quality  as  measured  by  the  instructor-to-trainee 
ratio,  training  program  throughput,  estimates  of 
resources  required  to  meet  training  objectives, 
and  training  program  costs.  Despite  the  current 
Army  basic  training  orientation  of  the  system,  it 
can  be  adapted  to  other  Army  training  programs 
and  to  training  programs  of  other  the  other  mili¬ 
tary  services  as  well. 

7.2  Summary  of  Results 

Experimental  results  from  testing  real-world 
scenarios  suggest  that  the  SPH-MPH  procedure 
can  determine  solutions  that  are  within  10%  to 
15%  of  the  "best"  (utopian)  solution  based  on  (7). 
These  results  were  obtained  in  reasonable  time 
on  a  486  microcomputer.  The  combination  of 
good,  timely  results  establishes  the  potential 
future  value  of  the  heuristic  to  basic  training 
management  and  related  decision  processes.  For 
the  one-period  time  lag  problem  the  heuristic 
achieved  results  that  were  91%  of  optimal  for  a 
small  but  representative  set  of  test  cases. 
However,  it  is  expected  that  the  scheduling  per¬ 
formance  of  the  heuristic  will  decline  relative  to 
exact  solution  methods  as  the  size  of  the  problem 
increases. 

Although  substantial  computational  effort  is 
necessary  to  generate  dynamic  programming 
solutions,  the  results  are  very  valuable.  First,  the 
quality  of  the  dynamic  programming  solutions  is 
superior  to  the  heuristic  results.  Second,  the 
dynamic  programming  algorithm  generates  the 
optimal  training  schedule,  the  optimal  objective 
function  value  for  each  schedule,  and  the  sched¬ 
uling  decisions  to  make  in  each  period  for  every 
initial  state.  Once  obtained,  the  results  are  easily 
stored  and  retrieved  using  a  simple  table  "look 
up"  procedure.  This  approach  is  ideal  for  quick¬ 
ly  generating  training  schedules  but  impractical 
for  "what-if"  analysis. 
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NOTES 

^  Major  Scott  Manderville,  then  of  the 
Directorate  of  Operations  and  Training  (DOT), 
Fort  Benning,  Georgia,  provided  cost  data  for 
the  study. 

^  SPH-MPH  notation  denotes  that  the  SPH  pro¬ 
cedure  generates  an  initial  feasible  training 
resource  schedule  that  serves  as  the  starting 
point  for  the  MPH  procedure. 
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